Google Researchers Propose TV Monitoring

Two Google research scientists want your computer to watch television with you so it can deliver personalized Internet content at the same time.

In a research paper presented last week at interactive television conference Euro ITV in Athens, Greece, Google researchers Michele Covell and Shumeet Baluja propose using ambient-audio identification technology to capture TV sound with a laptop PC to identify the show that is the source of the sound and to use that information to immediately return personalized Internet content to the PC.

"We showed how to sample the ambient sound emitted from a TV and automatically determine what is being watched from a small signature of the sound—all with complete privacy and minuscule effort," Covell and Baluja write on the Google Research Blog. "The system could keep up with users while they channel surf, presenting them with a real-time forum about a live political debate one minute and an ad-hoc chat room for a sporting event in the next."

The scheme is described in the research paper using a term that seems to be an oxymoron: mass personalization. It might also be characterized as a mixture of oil and water, a combination of television broadcasting and Internet narrowcasting.

"Mass-media channels typically provide limited content to many people," the paper explains, "the Web provides vast amounts of information, most of interest to few. ... Our goal is to combine the best of both worlds: integrating the relaxing and effortless experience of mass-media content with the interactive and personalized potential of the Web, providing mass personalization."

Not to mention massive profits. Marketers would kill to know exactly who's watching what when. With such a system, Google could extend its online dominance into television, and presumably radio, by offering advertisers unparalleled insight into the mass media audience.

The paper specifically contemplates the proposed system's potential as an advertising tool. "A similar procedure [to Google's online keyword bidding process] could be adapted to mass-personalization applications," the paper says. "Thus, content providers or advertisers might bid for specific television segments."

But making television less relaxing might be a hard sell. The couch potato evolved through a process of natural selection: TV viewers enjoy kicking back and zoning out. It's quite possible that mass personalization really is a contradiction in terms.

In time, however, Google may find a way to make it work. "This project is still in very early R&D stages and we don't have any specific product plans to announce at this time," Google spokesperson Sonya Boralv says in an e-mail.

"It's an interesting concept because we all sit with out laptops while watching TV," says Cynthia Brumfield, president of media research consultancy Emerging Media Dynamics. However, she balances her curiosity with some skepticism. Describing a media-multitasking scenario where she imagines watching TV while fielding phone calls, sending instant messages, and surfing the Internet, she admits, "It just seems awful to me."

According to Boralv, the system wouldn't be that intrusive. She writes, "If you were watching the news and wanted to delve deeper, this type of system could allow you to do that easily by automatically collecting related material and Web links for you. The beauty of the system that Michele and Shumeet describe is that it wouldn't be a distraction. If you don't want it you can ignore it and the PC browser will quietly update pages without bothering anyone—no input required and no audible output to form a distraction."

Those appalled by the prospect of Google tapping your television take heart: The proposal suggests user privacy would be respected. "[O]ur approach will not 'overhear' conversations," the paper says. "Furthermore, no one receiving (or intercepting) these statistics is able to eavesdrop, on such conversations, since the original audio does not leave the viewer's computer." Perhaps there's a lesson here for the National Security Agency.

There is, of course, an easier way to get audience data than listening with a laptop: Google could partner with telcos and cable companies in their respective efforts to deliver next-generation interactive television using Internet-style networking. The only problem is that IPTV, as the marriage of television and broadband networking is called, belongs to Microsoft. Microsoft is the major vendor of software for IPTV network operators and it has plans to be the Google of IPTV.

The trouble is, as Brumfield observes, "Google wants to be the Google of IPTV."

Given how wary of network operators like AT&T and Comcast are of Google, the company's best bet may be to make the Internet so appealing that it eclipses television completely. "What [this technology] might end up doing, if it were deployed and gained traction, is siphoning even more people away from the TV to the Internet," Brumfield says.