Follow This Blog: RSS feed
Neverending Search
Inside Neverending Search

Seeing AI: Leveraging artificial intelligence to better view the world

I’ve been writing about apps for a long time, but they are not of equal importance.

Microsoft’s free Seeing AI app may be a game changer for people with visual impairments.  The research project is designed to turn “turn the visual world into an audio experience,” narrating the world for those who cannot see it, in real time using artificial intelligence.

Available at the App StoreSeeing AI, is a talking camera experience. Introduced in late 2017, it is comprised of a growing variety of channels.

Seeing AI:

  • describes the people around you (including their emotions)
  • speaks text and signs as soon as they appear in front of your camera
  • scans and reads documents, books and letters, including handwriting by reading OCR and understanding formatting
  • identifies currency
  • scans barcodes on objects/products and adds product details when available. Seeing AI helps you locate the barcodes with audio beeps
  • describes images in other apps, for instance, email, Twitter, and WhatsApp
  • learns and recognizes your friends and describes other people around you, including their emotions
  • an experimental feature offers scene descriptions. Take a picture and the app will tell you about what the camera sees around you.
  • the app also generates an audible tone corresponding to your surrounding brightness and can perceive color

Screen Shot 2018-03-09 at 5.55.47 PM

These tutorial videos offer just a little peek into the Seeing AI experience:

On recognizing people

On hearing snippets of text

On reading documents

On locating barcodes and identifying products

On scene descriptions (an experimental channel to turn on in settings)

On currency identification (US currency is challenging because bills are all the same size!)

Though I cannot test this one out in a truly authentic way, I believe that Seeing AI is an important app for us to share. Please spread the word.

Joyce Valenza About Joyce Valenza

Joyce is an Assistant Professor of Teaching at Rutgers University School of Information and Communication, a technology writer, speaker, blogger and learner. Follow her on Twitter: @joycevalenza

Speak Your Mind