The Ultimate Search Engine - InformationWeek
Software // Enterprise Applications
02:30 PM
Connect Directly

The Ultimate Search Engine

Google, Microsoft, Yahoo and others are developing next-generation technologies that automate and personalize information search.

The search engine, that little browser tool into which you type a phrase, hit Enter, and hope for the best, is notoriously inefficient, often returning millions of off-the-mark URLs. People search for 11 minutes on average before finding what they're looking for, and half abandon searches without getting that far, according to Microsoft. By Gartner's estimate, half of potential Web sales are lost because visitors simply can't find what they want.

Google, Microsoft, Yahoo, and dozens of search specialists, including those catering to business customers, are racing to develop next-generation technologies that do a better job of getting people the information they seek. With emerging tools, people will no longer have to dumb down their queries with the pidgin language understood by first-generation search engines. They'll be able to ask questions in English and other languages--or pose no question at all and automatically receive results based on their earlier queries or the applications they're using.

The results users do get will include audio and video files, PowerPoint slides and other infographics, and structured data--all in one stream of results culled from the Web, PCs, and company databases. Over time, image searches will even detect information in the image itself, rather than by parsing metadata.

InformationWeek Download

Search results will be more accurate and automatically summarized, with relevance determined by individual preferences. New methods of presentation such as clustering, tag clouds, graphical scales that widen or narrow searches based on parameters, and automated categorization will make it easier to navigate results. And search engines will be enhanced by human intelligence and the wisdom of crowds through tagging, social bookmarking, and shared searches.

We won't have to wait long for some of these souped-up search engines. The following advanced capabilities are beginning to surface in a variety of places.

Most of today's search engines require a shorthand language some describe as keywordese. "It's kind of like talking to a 2-year-old," says Barney Pell, CEO of Powerset, a startup applying natural language processing to search. Over the next decade, Pell says, search engines will become more sophisticated in their ability to "understand meaning."

Today's search engines are just toddlers, says Barney Pell, CEO of Powerset

Today's search engines are just toddlers, says Pell
Powerset, Hakia, and other companies are developing search engines that apply linguistics--the science of language--to interpret questions, analyze Web content, and, as necessary, refine results through interaction with users. Hakia CEO Riza Berkan envisions search engines becoming "knowledgeable creatures in the future if we teach them how to talk and how to understand."

Semantic search engines parse language much like an English student does, using dictionaries and thesauri to interpret the meaning of words and link them using common rules of syntax and sentence structure. The sentence "IBM bought Tivoli for $743 million in 1996" includes concepts such as buying, buyer, subject of buy, year of buy, and purchase price.

For now, the process is aided by human beings who apply language rules and define categories to narrow searches, though Hakia's search engine can use language cues to find rough meaning in concepts it doesn't yet understand. "If it was fully automated, we would claim we have invented a human being," Berkan says. Web search engines like Google and Yahoo employ linguists, too, though they're not as far along with semantic search as Hakia or Powerset. Google's search engine can spell check and returns synonyms and variations of words, but it doesn't always answer questions accurately.

The technology of enterprise search company Autonomy powers the Federal Preservation Institute's Historic Preservation Learning Portal, a gateway to documents on preservation rules and methods. The institute uses semantic search to help nonexperts find information. "This allows them to ask in plain language questions that do not have the technical lingo that keywords may have," says Constance Ramirez, the institute's director. For example, a site visitor may ask about the preservation of red tile roofs in California. "It's really fascinating to see all the kinds of things that come back as relevant," says Ramirez.

IBM is working on specialized text analysis in fields such as health care and government. Customers use its OmniFind Analytics search engine to determine nuances like sentiment--whether a document reflects negatively or positively on a subject--and define and relate specialized words, concepts, and proper nouns used inside a company.

1 of 4
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
[Interop ITX 2017] State Of DevOps Report
[Interop ITX 2017] State Of DevOps Report
The DevOps movement brings application development and infrastructure operations together to increase efficiency and deploy applications more quickly. But embracing DevOps means making significant cultural, organizational, and technological changes. This research report will examine how and why IT organizations are adopting DevOps methodologies, the effects on their staff and processes, and the tools they are utilizing for the best results.
Register for InformationWeek Newsletters
White Papers
Current Issue
Digital Transformation Myths & Truths
Transformation is on every IT organization's to-do list, but effectively transforming IT means a major shift in technology as well as business models and culture. In this IT Trend Report, we examine some of the misconceptions of digital transformation and look at steps you can take to succeed technically and culturally.
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll