Do We Expect Too Much from Data? - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management
06:44 AM
Doug Henschen
Doug Henschen
Connect Directly

Do We Expect Too Much from Data?

We all like to believe in "fact-based decision making," but noted author John Allen Paulos makes the case that our increasingly data-driven culture can be easily mislead by numbers.

The subtitle/question "do we expect too much from data?" appeared in The New York Times Magazine yesterday atop the article "The Way We Live Now: Metrics Mania," by noted author John Allen Paulos. The columnist and Temple University methematics professor has written, among other books, Innumeracy and A Mathematician Reads the Newspaper, both of which help non-mathematicians make better sense of data. Statistics can be perplexing enough, but in his article, Paulos makes the case that our increasingly data-driven culture can lose sight of context and be easily mislead. "Unless we know how things are counted," he writes, "we don't know if it's wise to count on the numbers."Anybody who follows politics knows that it's all too easy for both sides to come up with convincing-sounding numbers to make their case. "By appropriate choices of criteria , measurement protocols and weights, almost any desired outcome can be reached," writes Paulos in describing this phenomenon.

We've seen an a disturbing twisting of data over the last few weeks as the oil industry, the U.S. government and environmentalists each offer their own estimates of the extent of the oil spill off Louisiana. There's no question that it's a bad situation, but depending on whose figures you believe, we're dealing with something that's either half as big as the Exxon Valdez oil spill in Alaska, three times bigger than that spill, or a catastrophe that may ultimately be six times larger than the Exxon Valdez spill.

Paulos offers a medical-research example in which a five-year survival rate for a particular disease is 100% in one region and 0% in another, yet the latter region has an equally effective and cheaper approach to treating the disease. How is this possible? Paulos explains:

Suppose that whenever people contract the disease, they always get it in their mid-60s and live to the age of 75. In the first region, an early screening program detects such people in their 60s. Because these people live to age 75, the five-year survival rate is 100 percent. People in the second region are not screened and thus do not receive their diagnoses until symptoms develop in their early 70s, but they, too, die at 75, so their five-year survival rate is 0 percent. The laissez-faire approach thus yields the same results as the universal screening program, yet if five-year survival were the criterion for effectiveness, universal screening would be deemed the best practice.

The point here is not that we should give up on data and go back to gut-level decision-making. As Paulos concludes, we should use data "with as much care and wisdom as we can muster."We all like to believe in "fact-based decision making," but noted author John Allen Paulos makes the case that our increasingly data-driven culture can be easily mislead by numbers.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Get Your Enterprise Ready for 5G
Mary E. Shacklett, Mary E. Shacklett,  1/14/2020
Modern App Dev: An Enterprise Guide
Cathleen Gagne, Managing Editor, InformationWeek,  1/5/2020
9 Ways to Improve IT and Operational Efficiencies in 2020
Cynthia Harvey, Freelance Journalist, InformationWeek,  1/2/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Flash Poll