Can The Jeopardy Challenge Help IBM Compete With Apple And Google? - InformationWeek
IoT
IoT
Software // Information Management
Commentary
2/14/2011
01:39 PM
Doug Henschen
Doug Henschen
Commentary
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%
RELATED EVENTS
Faster, More Effective Response With Threat Intelligence & Orchestration Playboo
Aug 31, 2017
Finding ways to increase speed, accuracy, and efficiency when responding to threats should be the ...Read More>>

Can The Jeopardy Challenge Help IBM Compete With Apple And Google?

Noted author Stephen Baker says this week's man vs. machine TV contest will help Big Blue attract the best and brightest. Early uses of the technology will show up on cell phones, he predicts.

Stephen Baker already knows the outcome of this week's Jeopardy quiz-show challenge between IBM's Watson super computer and grand champions Ken Jennings and Brad Rutter. But he won't say who (or what) comes out on top.

Like everyone present during the taping of the shows (to be broadcast on February 14, 15 and 16), Baker signed what he describes as a "Draconian" non-disclosure agreement. What sets Baker apart from everyone else in the audience that day is the fact that he will reveal the in-depth, behind-the-scenes story of Watson in great detail.

Author of the notable 2008 big-data-computing expose "The Numerati," Baker has spent the last year researching and writing Final Jeopardy: Man Vs. Machine and the Quest to Know Everything, set for release on February 17, the day after the matches conclude. In fact, the first 11 chapters are already available as an e-book, with the 12th and final chapter to be downloadable the day the print edition is released.

Baker couldn't discuss the outcome of the matches. But in a pre-show interview with Information Week, he shared details on the wake-up call IBM had in the early days of the project and the disagreements the company had with Jeopardy's producers. Baker also shares his take on why the "contrivance" of a TV quiz-show challenge is a serious matter for IBM and what's likely to come of the technology breakthroughs.

A Darwinian Experiment

You might think IBM could assemble all the technology needed to run Watson from its vast portfolio of commercial computing technologies, but Baker says that's not the case. Sure, it runs on off-the-shelf IBM Power 750 Servers, but much of the analytic software -- the heart of Watson's brain -- was developed from scratch.

Back in 2007, when the project was in its opening stages at IBM Research, David Ferrucci, the computer scientist leading the challenge, feared that the project would either fail miserably or, worse, be discovered and outdone by some basement hacker taking a novel approach.

To set up internal competition that would subject his researchers to "Darwinian pressures," as Baker writes in his book, Ferrucci assigned James Fan, a recent doctoral grad and new researcher on his team, to develop a system that would take on IBM's best incumbent question and answer technology, a platform called Piquant.

Inside Watson, IBM's Jeopardy Computer
(click image for larger view)
Slideshow: Inside Watson, IBM's Jeopardy Computer

The two rival teams, Fan and a group of veteran researchers experienced with Piquant, were given four weeks to develop and train a system on the same set of 500 sample Jeopardy clues. The Piquant developers could draw from an existing platform based on years of research, but Fan had to cobble a system together completely from scratch. To do so, he combined entity extraction technologies (software that can spot people, places, things, dates and concepts) with lots of algorithms -- some sophisticated, some incredibly crude.

In one example, he developed a program that would submit the Jeopardy clue as a Google search and then take the title of the first Wikipedia page appearing in the result set as the correct answer.

Previous
1 of 3
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
IT Strategies to Conquer the Cloud
Chances are your organization is adopting cloud computing in one way or another -- or in multiple ways. Understanding the skills you need and how cloud affects IT operations and networking will help you adapt.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll