Google Translate App's Sign-Reading Capabilities Expanded

Travelers can now translate a greater variety of languages in images more efficiently, even when offline. Foreign travelers in remote areas with poor connectivity now have broader communication options.

Thomas Claburn, Editor at Large, Enterprise Mobility

July 30, 2015

3 Min Read
<p style="text-align:left">(Image: Google)</p>

Windows 10: 10 Things To Know At Launch

Windows 10: 10 Things To Know At Launch

Windows 10: 10 Things To Know At Launch (Click image for larger view and slideshow.)

The Google Translate app has been accelerated and expanded to support realtime in-camera translation of 27 languages, up from seven in January, with the help of dirty letters and a self-contained neural network. The in-camera translations are handy when trying to ferret out the meaning of road signs or menu pages while traveling in a foreign country, for example.

Google Translate's text and speech translation capabilities still require a network connection to Google's data centers. But the app's longstanding ability to recognize and translate letters contained in images -- in realtime or in a snapshot -- while untethered from the cloud has become more efficient and more comprehensive. Foreign travelers in remote areas with poor connectivity now have broader communication options.

Google has managed this feat through sophisticated programming techniques and densely packed data. As Google software engineer Otavio Good explained in a blog post, the company created a small neural network inside the app.

"If you're translating a foreign menu or sign with the latest version of Google's Translate app, you're now using a deep neural net," said Good. "And the amazing part is it can all work on your phone, without an Internet connection."

Convolutional networks have proven to be effective for image recognition in recent years. But such networks often rely on large data sets stored in the cloud or local server banks. The CamFind app, for example, relies on neural networks and crowdsourcing for image recognition. But it requires a network connection to operate.

To create a compact neural network capable of recognizing the highly variable shapes of letters across multiple languages, Google engineers relied on sophisticated math and speedy SIMD processor instructions. To train the neural network, they created data sets using letters obscured by dirt.

"Letters out in the real world are marred by reflections, dirt, smudges, and all kinds of weirdness," explained Good. "So we built our letter generator to create all kinds of fake 'dirt' to convincingly mimic the noisiness of the real world -- fake reflections, fake smudges, fake weirdness all around."

Google introduced instant bidirectional image translation in Google Translate in January for English, French, German, Italian, Portuguese, Russian, and Spanish. Wednesday's update adds Bulgarian, Catalan, Croatian, Czech, Danish, Dutch, Filipino, Finnish, Hungarian, Indonesian, Lithuanian, Norwegian, Polish, Romanian, Slovak, Swedish, Turkish, and Ukrainian. In addition, the app can handle one-way image translation from English to Hindi or Thai. The app supports these 27 languages for realtime image translation, and 37 languages using stored snapshots.

Google previously made image recognition available in its Google Goggles app for Android in December 2009. Google Goggles was subsequently released for iOS and then discontinued for iOS in 2014. Around the same time, Google acquired Word Lens, an image recognition and translation app, that was folded into Google Translate in January.

About the Author(s)

Thomas Claburn

Editor at Large, Enterprise Mobility

Thomas Claburn has been writing about business and technology since 1996, for publications such as New Architect, PC Computing, InformationWeek, Salon, Wired, and Ziff Davis Smart Business. Before that, he worked in film and television, having earned a not particularly useful master's degree in film production. He wrote the original treatment for 3DO's Killing Time, a short story that appeared in On Spec, and the screenplay for an independent film called The Hanged Man, which he would later direct. He's the author of a science fiction novel, Reflecting Fires, and a sadly neglected blog, Lot 49. His iPhone game, Blocfall, is available through the iTunes App Store. His wife is a talented jazz singer; he does not sing, which is for the best.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights