Stephen Wolfram's Answer To Google

Google has a lot of would-be competitors, but no startup to date has put forth a credible challenge as a general-purpose Internet search engine. Only Google, Microsoft, Yahoo, and, perhaps, Ask qualify for the search major league.

If and when Google is surpassed as a method for finding online information, it will be by a company that manages to come up with a better way to search. It won't be an improved search engine; it will be something that redefines the act of research itself.

Wolfram/Alpha could be that thing. Or it could the heir to the over-hyped "Ginger," the project that was supposed to change the world and turned out to be clever but short of revolutionary: the gyrostabilized Segway scooter.

But let's allow for the moment the possibility that it could change the search game.

Wolfram/Alpha is a computational knowledge engine. It's being developed by respected computer scientist Stephen Wolfram, creator of Mathematica, to compute the answer to any factual question.

With billions of pages of human knowledge online, we can conduct very efficient searches for terms and phrases in the text, Wolfram explained in a blog post.

"But we can't compute from that," he said. "And in effect, we can only answer questions that have been literally asked before. We can look things up, but we can't figure anything new out."

One way researchers have been trying to make search engines smarter is semantic search, an effort to help search engines understand the meaning of search queries. While semantic technology can be used in limited circumstances to make search more effective, it falls short as a general way to understand natural language in any context.

Semantic search requires the construction of ontologies, which explicitly define what terms mean and how the terms are related to each other. Because these ontologies must be defined in advance, semantic systems are only as smart as they have been taught to be. But applied to specific situations, semantic technology can help make search queries less ambiguous and provide access to information that might otherwise have been missed.

Nova Spivak, who has been developing semantic search technology to power a Web site called Twine, sees a lot of potential in Wolfram's project. He suggests it may be as important to the world as Google, though he sees it complementing Google rather than replacing it.

"It doesn't simply return documents that (might) contain the answers, like Google does, and it isn't just a giant database of knowledge, like the Wikipedia," he explained in a post on Twine. "It doesn't simply parse natural language and then use that to retrieve documents, like Powerset, for example. Instead, Wolfram Alpha actually computes the answers to a wide range of questions -- like questions that have factual answers such as 'What is the location of Timbuktu?' or 'How many protons are in a hydrogen atom?,' 'What was the average rainfall in Boston last year?,' 'What is the 307th digit of Pi?,' 'Where is the ISS?' or 'When was GOOG worth more than $300?'"

Wolfram/Alpha won't be available to the public until May. If it works as advertised, it will be able to do something Google can't: provide answers that don't already exist in indexed documents.

"I wasn't at all sure it was going to work," Wolfram said. "But I'm happy to say that with a mixture of many clever algorithms and heuristics, lots of linguistic discovery and linguistic curation, and what probably amount to some serious theoretical breakthroughs, we're actually managing to make it work."

Stay tuned.

What does the ultimate search engine look like? InformationWeek has published an independent analysis of this topic. Download the report here (registration required).