Seeing AI from Microsoft Can Identify Almost Anything

Originally available only on smart glasses, Microsoft’s Seeing AI app is now available on iOS and actively describes what the phone camera captures in real time.

In keeping with Microsoft CEO Satya Nadella’s promise at Build 2017 to go all in on artificial intelligence and genuine efforts to create applications for all platforms, the company’s newest app Seeing AI is available on iOS devices. Seeing AI was developed to help the visually impaired recognize things without seeing them using the camera of the iPhone and iPad. When the app senses an object (person, place, or thing), an audible voice states what it “sees” to help the visually impaired person get a better sense of his or her surroundings as well as assist in tasks such as reading.

Seeing AI can not only identify random things, but it can learn the faces of people you know and even tell you information about them, such as their age range and if they are smiling. Here are the channels available through Seeing AI:

Short Text

The Short Text channel allows your mobile device camera can identify groups of words or phrases, ranging from a title on a door to a name on an envelope.

Document

If you need to read an entire page instead of only a few words, this channel will read and scan in the full text of a document and read back the separate section to you.

Product

To get an accurate description of a product you have in front of you, simply hover the camera of the bar code to scan it and get the product name. If you are having trouble finding the bar code, the device will begin beeping when it senses part of the bar code and will increase in speed as you reveal more of the bar code.

Person

One of the most useful channels, you can take a photo of a person and get a relatively accurate description of the person’s gender, age, and facial characteristics such as glasses and a smile. The camera will let you know if the person is in the frame and how far he or she is from you. Additionally, you can teach Seeing AI to recognize a person in the future by inputting his or her name.

Scene

One of the more experimental channels is Scene where the app will attempt to determine what is display on the camera, such as “man standing at desk” or “woman sitting on a couch using a laptop.”

Currency

Also new and currently only works with US dollars, the camera will be able to recognize the bill note amount, such as $1, $5, $20, etc.

Recognize Images in Other Apps

If you are using another application such as social media or photo sharing, Seeing AI can read the image text and describe the image to you using an option from the pull-up menu on iOS devices.

Seeing AI is available now for iPhone and iPad with an Android or Windows phone version yet to be announced. Check out Microsoft’s video playlist for demos of each channel for Seeing AI.