Facebook's AI Technology Describes Photos To Visually Impaired Users - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Social
11:05 AM

Facebook's AI Technology Describes Photos To Visually Impaired Users

Visually impaired Facebook users will now find it easier to catch up with their friends' and family's photos with automatic alternative text for iOS. The new feature is based on the social network's AI technology.

Mobile Messaging Apps: 8 Tips For Keeping Your Workplace Secure
Mobile Messaging Apps: 8 Tips For Keeping Your Workplace Secure
(Click image for larger view and slideshow.)

Facebook is reaching out to the more than 280 million visually impaired people on the planet with a new technology called automatic alternative text, which promises to help the blind community experience the social network in the same way others enjoy it.

Automatic alternative text, or automatic alt text, is a described by Facebook engineers as a new development that generates a description of a photo using advancements in object recognition technology. The company detailed the technology in an April 4 blog post written by Shaomei Wu, a Facebook software engineer; Hermes Pique, software engineer on iOS; and Jeffrey Wieland, Facebook's head of accessibility.

The reason that an Apple engineer is involved is that automatic alt text will be first available on iOS screen readers set to English. However, the blog post notes that Facebook plans to add this functionality for other languages and platforms soon.

People using screen readers on iOS devices will hear a list of items a photo may contain as they swipe past photos on Facebook. Previously, people using screen readers would only hear the name of the person who shared the photo, followed by the term "photo" when they came upon an image in their News Feed.

Facebook's Accessibility Team
(Image: Facebook)

Facebook's Accessibility Team

(Image: Facebook)

Facebook used its own artificial intelligence (AI) technology to create automatic alt text. According to the post:

This is possible because of Facebook's object recognition technology, which is based on a neural network that has billions of parameters and is trained with millions of examples. Each advancement in object recognition technology means that the Facebook Accessibility team will be able to make technology even more accessible for more people. When people are connected, they can achieve extraordinary things as individuals and as a community -- and when everyone is connected, we all benefit.

While very much still in the experimental phase, the technology is also intensely personal for one of the engineers.

Create a culture where technology advances truly empower your business. Attend the Leadership Track at Interop Las Vegas, May 2-6. Register now!

Matt King, who is blind, helped work on the project along with Wieland, who leads the accessibility team, which is part of the user interface engineering group. 

The group's goal is to improve Facebook for people with disabilities, and ensure that the company's products cooperate with assistive technology.

"The fact that you have somebody who has worked on accessibility who actually has the disability, is in a leadership position at probably the most pervasive application on the planet and is willing to put themselves out there like that," Rich Schwerdtfeger, CTO of accessibility at IBM and a former colleague of King, told Mashable. "That's a big deal."

Facebook's announcement follows a recent initiative by Twitter that aims to make it easier for visually impaired people to use that social media platform.

People using Twitter's iOS and Android apps can add descriptions -- also known as alternative text (alt text) -- to images in Tweets.

When a user adds an image to a Tweet, each thumbnail in the composer will have an "add description" button. Users can then tap it to add a description to the image.

People who are visually impaired will have access to the description through assistive technology -- such as screen readers and braille displays. The descriptions can be up to 420 characters.

While so much of today's information highway leans towards text and visually based technology, major corporations have been reaching out to the visually impaired in an effort to foster a community of inclusion.

A company best known for affordable cars is branching out in an unexpected direction, as Toyota announces Project BLAID, a wearable device for the blind and visually impaired.

Worn around the user's shoulders, the device is designed to help blind and visually impaired people better navigate indoor spaces, such as office buildings and shopping malls, by helping them identify everyday features, including restrooms, escalators, stairs, and doors.

Nathan Eddy is a freelance writer for InformationWeek. He has written for Popular Mechanics, Sales & Marketing Management Magazine, FierceMarkets, and CRN, among others. In 2012 he made his first documentary film, The Absent Column. He currently lives in Berlin. View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

10 Things Your Artificial Intelligence Initiative Needs to Succeed
Lisa Morgan, Freelance Writer,  4/20/2021
Tech Spending Climbs as Digital Business Initiatives Grow
Jessica Davis, Senior Editor, Enterprise Apps,  4/22/2021
Optimizing the CIO and CFO Relationship
Mary E. Shacklett, Technology commentator and President of Transworld Data,  4/13/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll