Microsoft's Seeing AI app helps describe the world to the visually impaired

Posted July 13, 2017

The Redmond company revealed the effort past year at its Build developer show, demonstrating how a more developed version might help Saqib Shaikh, a Microsoft engineer on the team who lost his sight at age 7. Seeing AI is now available to download for free on iOS, but only in the US. With the app, you can simply focus your phone's camera on an object or person, and then you'll get a narration about what it is or who they are and even how they're feeling. On top of that, it can quickly recognize and read short text snippets, as well as full documents and much more.

Microsoft's Seeing AI app can also work with other apps too. As you point the camera at things in the world around you, the app will describe that world in a quick, informative manner. The app is able to recognize people as long as they're part of the users saved friends.

The Seeing AI app is very simple to use. One example shown off by Microsoft is that it can describe images on Twitter. This is the same kind of tech being used by other companies to power self-driving cars and drones, as pointed out by The Verge. After scanning a product's barcode, in some cases you'll receive additional information about the product beyond just its name.

Microsoft has a short introductory video that gives a great taste of all that the app can do, embedded below. The first experimental feature is scene descriptions. Most recently it added support for Apple's MacBook Pro Touch Bar in Office apps.

Microsoft said Seeing AI demonstrates how technology can make the world more inclusive, and we couldn't agree more.

Another useful feature here is that the Seeing AI app can also scan/read documents and read signs for users. The company didn't say whether a version would be released for Android phones.