Seeing AI: Leveraging artificial intelligence to better view the world

I’ve been writing about apps for a long time, but they are not of equal importance.

Microsoft’s free Seeing AI app may be a game changer for people with visual impairments.  The research project is designed to turn “turn the visual world into an audio experience,” narrating the world for those who cannot see it, in real time using artificial intelligence.

Available at the App Store, Seeing AI, is a talking camera experience. Introduced in late 2017, it is comprised of a growing variety of channels.

Seeing AI:

• describes the people around you (including their emotions)
• speaks text and signs as soon as they appear in front of your camera
• scans and reads documents, books and letters, including handwriting by reading OCR and understanding formatting
• identifies currency
• scans barcodes on objects/products and adds product details when available. Seeing AI helps you locate the barcodes with audio beeps
• describes images in other apps, for instance, email, Twitter, and WhatsApp
• learns and recognizes your friends and describes other people around you, including their emotions
• an experimental feature offers scene descriptions. Take a picture and the app will tell you about what the camera sees around you.
• the app also generates an audible tone corresponding to your surrounding brightness and can perceive color

On scene descriptions (an experimental channel to turn on in settings)

On currency identification (US currency is challenging because bills are all the same size!)

Though I cannot test this one out in a truly authentic way, I believe that Seeing AI is an important app for us to share. Please spread the word.

Filed under: accessibility, apps, visually impaired