SCROLL DOWN TO READ THE POST
Seeing AI: Leveraging artificial intelligence to better view the world
I’ve been writing about apps for a long time, but they are not of equal importance.
Microsoft’s free Seeing AI app may be a game changer for people with visual impairments. The research project is designed to turn “turn the visual world into an audio experience,” narrating the world for those who cannot see it, in real time using artificial intelligence.
- describes the people around you (including their emotions)
- speaks text and signs as soon as they appear in front of your camera
- scans and reads documents, books and letters, including handwriting by reading OCR and understanding formatting
- identifies currency
- scans barcodes on objects/products and adds product details when available. Seeing AI helps you locate the barcodes with audio beeps
- describes images in other apps, for instance, email, Twitter, and WhatsApp
- learns and recognizes your friends and describes other people around you, including their emotions
- an experimental feature offers scene descriptions. Take a picture and the app will tell you about what the camera sees around you.
- the app also generates an audible tone corresponding to your surrounding brightness and can perceive color
These tutorial videos offer just a little peek into the Seeing AI experience:
On recognizing people
On hearing snippets of text
On reading documents
On locating barcodes and identifying products
On scene descriptions (an experimental channel to turn on in settings)
On currency identification (US currency is challenging because bills are all the same size!)
Though I cannot test this one out in a truly authentic way, I believe that Seeing AI is an important app for us to share. Please spread the word.
About Joyce Valenza
Joyce is an Assistant Professor of Teaching at Rutgers University School of Information and Communication, a technology writer, speaker, blogger and learner. Follow her on Twitter: @joycevalenza
SLJ Blog Network