Prediction Markets and Unpredictable Decision Making

Prediction markets are mathematically based but human powered, a tool for turning collective human insights into forecasts. The approach enables individuals to bet on ideas or events. Twists such as anonymous participation, restriction to experts, and pre-screening for character traits are designed to reduce bias and boost accuracy.

The New York Times reports that "companies use prediction markets to funnel ideas from the work force" (Betting to Improve the Odds, April 9, 2008). Steve Lohr's article provides illuminating examples from companies including Best Buy, InterContinental Hotels, and Hewlett-Packard. Lohr also notes, "for years, public prediction markets have been used for politics, where buyers and sellers bet on which candidate will win a particular race." It's the kind of article that everyone interested in decision sciences should read, and then follow-up on to understand not only the mechanics of the techniques but also their limitations.I was surprised, however, that Lohr either didn't know about or chose not to cite the Pentagon's Policy Analysis Market (PAM), an ill-fated 2003 initiative. As I described in a 2003 Intelligent Enterprise column, "the program featured a Web-mediated futures market on Middle East politics and sought to apply established forecasting techniques to predicting terrorism." PAM was killed very soon after launch because it allowed politically unpalatable betting on the likelihood of events such as the overthrow of the Jordanian monarchy.

Corporate prediction markets face the same limits as political ones. Choices put before participants may be over-constrained or badly posed. The methods cannot forecast unconsidered possibilities. And they offer little of the explanatory power that would allow general rules to be derived from a given prediction. Most difficult of all, perhaps, is that decision-making power is often held by individuals who act for reasons that are neither understood nor considered by the people who are participating in the prediction exercise. Read Maureen Dowd's New York Times column, Toil and Trouble, published the same day as Lohr's article, where she describes "predictably for everyone except the chuckleheaded."

Crowds may (sometimes) be wise, but they don't make decisions. Rather, decisions are often made by individuals whose perspectives are at odds with those of the kind of people who would participate in a prediction-markets exercise. Real-world decisions may also be the product of an arcane system that embodies values and principles that are not (or no longer) broadly accepted. Consider the U.S. Electoral College, designed for a federal republic rather than for a popular democracy.

Prediction markets are a promising decision-support tool. But participation in groupthink forecasting should not be seen as equivalent to making and being accountable for decisions.Prediction markets are mathematically based but human powered, a tool for turning collective human insights into forecasts. The approach enables individuals to bet on ideas or events. The New York Times reports that "companies use prediction markets to funnel ideas from the work force." It's the kind of article that everyone interested in decision sciences should read, and then follow-up on to understand not only the mechanics of the techniques but also their limitations.