Inside Watson, IBM's Jeopardy Computer
IBM has spent four years and untold hundreds of millions of dollars developing Watson, a computer that can play Jeopardy. In fact, Watson answers questions so quickly and accurately that IBM challenged Jeopardy and two of the show's all-time champions to a match. Jeopardy's producers and the human contestants agreed. Taping took place January 14, and the three computer-vs.-human episodes will air February 14-16. Cynics may call it a publicity stunt, but the project has brought real advances in c
![](https://eu-images.contentstack.com/v3/assets/blt69509c9116440be8/bltc0182b2356ae8eed/64b83949410a1b4c0bd7459b/IW_generic_image.png?width=700&auto=webp&quality=80&disable=upscale)
Meet Watson, a supercomputer designed to play Jeopardy. Watson is what IBM calls a "deep question and answer" application that runs on 10 racks of commercially available IBM Power 750 Servers. It's essentially a Jeopardy-optimized system on a massively parallel processing (MPP) platform that performs at 80 teraflops, or about 80 trillion operations per second. Watson has a 15-terabyte memory, but it's not a database. Rather, Watson was trained in advance on some 200 million pages of content -- on topics ranging from history and science to politics and pop culture. The resulting content store is based on IBM's (now open source) UIMA (Unstructured Information Management Architecture) with entities such as people, places, things, dates, and concepts meta-tagged for retrieval. When Watson is presented with a Jeopardy clue -- submitted as a text file when the clue is revealed -- search technology retrieves thousand of possible answers. Thousands of algorithms (the real "secret sauce") developed by IBM Research then score the possible answers and measure the confidence in best answer available. If confidence is high, Watson will ring in -- through a mechanical actuator much like the one human contestants use.
SEE ALSO:
"The hard part for the computer is finding and justifying the correct answer," says Dr. David Ferrucci, the "Principal Investigator" at IBM Research who led the Watson project. "For each of thousand of plausible answers, Watson gathers evidence and uses thousands of algorithms to understand what's most likely to be the right answer." Most of the effort in the four-year project involved developing the algorithms.
SEE ALSO:
This graphic shows the progress that had to be made in developing Watson. The brown line shows the computer's initial performance playing Jeopardy circa 2007. The blue dots show the performance of Jeopardy champions and the red dots the performance of grand champions. Over four years, the Watson team steadily improved the computer's performance up to grand-champion levels.
SEE ALSO:
Watson can't look things up on the Internet. That would be unfair! Instead, the computer has a fixed, 15-terabyte content store that is "trained" on the equivalent of one million books on diverse topics. The content goes through an analysis stage in which entities such as people, places, things, dates and concepts are marked with meta tags. Search technology retrieves contextually appropriate information quickly, but the real secret behind Watson is its battery of analytic algorithms used to decipher the clue and score the computer's confidence in having the right answer. This application takes two hours to come up with correct answers when running on a single compute node, but that time drops to less than three seconds when running on thousands of nodes available on IBM's massively parallel processing platform.
SEE ALSO:
IBM says Watson-like computers could take on challenges in many domains. "I can't imagine a single industry where there isn't potential to transform it, whether it's the healthcare system, the legal system or any area where time-sensitive and accuracy-sensitive responses are needed," said Dr. John E. Kelly, III, senior vice president and director of IBM Research.
SEE ALSO:
Is the Jeopardy IBM Challenge an enormous publicity stunt? Clips on YouTube have already generated hundreds of thousands of hits. And as seen here, more than a dozen television news crews were at a January 13 press conference along with scores of journalists, analysts and bloggers. That's just the beginning. PBS is planning a Nova special entitled "Smartest Machine on Earth: Can a Computer Win on Jeopardy?" But if Watson can't go toe-to-toe with Jeopardy champions, Big Blue runs the risk of exhibiting very red faces.
SEE ALSO:
As he explains in this YouTube video, Jeopardy's long-time Executive Producer, Harry Friedman, said the show would not have agreed to take part in a stunt or a gimmick. Here at a January 13 press conference, Friedman announces that IBM will donate 100% of Watson's winnings to charity. Human contestants Ken Jennings and Brad Rutter have agreed to give half of their winnings to charity. The winner gets $1 million. The second-place finisher gets $300,000. Third place nets $200,000.
SEE ALSO:
Watson, in the center represented by an Avatar on a computer monitor, will play against Ken Jennings, left, winner of 74-consecutive-games during the 2004-05 season, and Brad Rutter, the $3.26 million all-time-record money winner on Jeopardy. Watson has to trigger a mechanical actuator (seen in front of the monitor) just as the human contestants do when they're ready to offer an answer posed in the form of a question.
SEE ALSO:
Jennings, Watson, and Rutter squared off in a two-minute test round held for the press on January 13. The contestants taped three episodes on January 14 at a temporary studio built at IBM's T.J. Watson Research Center in Yorktown Heights, NY.
SEE ALSO:
The categories and clues used in the test round and the games taped for television are a closely guarded secret. The show's producers also removed any topics previously encountered by any of the contestants (including Watson) in prior games or test rounds. Clues are submitted to Watson in the form or a text file the moment they are revealed to the human contestants. The "answer" buttons are disabled until Alex Trebek finishes reading the entire question.
SEE ALSO:
In the early going during the press test round, Watson quickly answered the first question and then rifled through all but the last clue in the category. The computer drew a big laugh when its gentle, male voice said, "Let's finish 'Chicks Dig Me'." Watson's confidence in potential answers is shown at the bottom of the screen on studio monitors (unseen by the human contestants). Orange indicates low confidence. Watson did not attempt to answer this question, though the correct answer was, indeed, "Neanderthal."
SEE ALSO:
Watson had the correct answer -- "Who is Michael Caine?" -- to the clue "He's worth every penny as Alfred Pennyworth in a 'The Dark Knight'," but Ken Jennings beat the computer's response time. The category was "M.C. 5," so Watson clearly understood that the answer had to fit the "M.C." mold. That said, one has to wonder why "Major Critical" and "Marine Corps" were even considered?
SEE ALSO:
Despite Watson's early lead, Jennings closed the gap, doing well on the "Children's Book Titles" category. Watson finished the two-minute round with $4,400 to Jennings' $3,400 and Rutter's $1,200.
SEE ALSO:
Looking dapper as ever at the Jeopardy IBM Challenge press conference, Alex Trebek, Jeopardy's host for 27 years, said he's hoping one of the humans will win. Asked if he will have personal interactions with Watson at the start of the Double Jeopardy round, Trebek said, "Watson can neither hear nor see, so I will not ask Watson any questions whatsoever. I will probably try to have fun at its expense."
SEE ALSO:
Trebek moderated a press conference with, from left to right, executive producer Harry Friedman, contestants Brad Rutter and Ken Jennings, and David Ferrucci and John Kelly of IBM Research. One journalist asked if Watson could be compared to the fictional HAL 9000 computer in 2001: A Space Odyssey, but Ferrucci said the development team took inspiration from the question-answering ship's computer featured in the television series Star Trek.
SEE ALSO:
Rutter and Jennings had fun answering questions about the looming human-versus-machine showdown. "When Watson's progeny comes back from the future to kill me, I have my escape route planned," said Rutter, in a tongue-in-cheek reference to the Terminator movies. Asked how he managed to win 74 consecutive Jeopardy games, Jennings quipped, "I owe it all to Wikipedia." He then answered seriously that he and other top quiz show contestants seem to have sponge-like associative memory capabilities.
SEE ALSO:
Set your DVRs for the Jeopardy episodes airing February 14, 15, and 16, 2011. The three shows were all taped at the IBM T.J. Watson Research Center on January 14, but outcome is a secret until the shows air. The PBS Nova special "Smartest Machine on Earth: Can a Computer Win on Jeopardy?" airs February 9. And Stephen Baker's book, Final Jeopardy: Man vs. Machine and the Quest to Know Everything, hits stores February 17."
SEE ALSO:
Set your DVRs for the Jeopardy episodes airing February 14, 15, and 16, 2011. The three shows were all taped at the IBM T.J. Watson Research Center on January 14, but outcome is a secret until the shows air. The PBS Nova special "Smartest Machine on Earth: Can a Computer Win on Jeopardy?" airs February 9. And Stephen Baker's book, Final Jeopardy: Man vs. Machine and the Quest to Know Everything, hits stores February 17."
SEE ALSO:
Meet Watson, a supercomputer designed to play Jeopardy. Watson is what IBM calls a "deep question and answer" application that runs on 10 racks of commercially available IBM Power 750 Servers. It's essentially a Jeopardy-optimized system on a massively parallel processing (MPP) platform that performs at 80 teraflops, or about 80 trillion operations per second. Watson has a 15-terabyte memory, but it's not a database. Rather, Watson was trained in advance on some 200 million pages of content -- on topics ranging from history and science to politics and pop culture. The resulting content store is based on IBM's (now open source) UIMA (Unstructured Information Management Architecture) with entities such as people, places, things, dates, and concepts meta-tagged for retrieval. When Watson is presented with a Jeopardy clue -- submitted as a text file when the clue is revealed -- search technology retrieves thousand of possible answers. Thousands of algorithms (the real "secret sauce") developed by IBM Research then score the possible answers and measure the confidence in best answer available. If confidence is high, Watson will ring in -- through a mechanical actuator much like the one human contestants use.
SEE ALSO:
About the Author(s)
You May Also Like