6 Causes Of Big Data Discrepancies - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Big Data Analytics
07:06 AM
Lisa Morgan
Lisa Morgan
Connect Directly

6 Causes Of Big Data Discrepancies

The same data can yield wildly different results. Here are some of the reasons for these fascinating, frustrating, or even dangerous discrepancies.
1 of 7

(Image: Geralt via Pixabay)

(Image: Geralt via Pixabay)

As the universe of big data continues to explode, organizations struggle to identify and leverage the data that matters most. As companies continue to add more data sources to the mix, the number of potential data sets grows, as do the opportunities for new and different insights. Because data can be combined in so many different ways, there are many possible outcomes.

"People have different choices from the same data set," said Kirk Borne, principal data scientist at Booz Allen Hamilton in an interview. "People will choose different things they think are informative, so the search is to find the most important variables."

As a result, the same data can result in very different interpretations.

"If I run a supernova simulation where the resolution is too low and two supernova scientists analyze that, if one knows the simulation is not a sufficient resolution and the other doesn't, they would come to very different conclusions," said Tony Mezzacappa, chair of theoretical and computation astrophysics at the University of Tennessee, in an interview. "Data completeness is part of data quality. People should understand what the dangers are in extracting conclusions based on such data."

Whether or not data is complete enough may not be obvious until later. For example, cosmic microwave background radiation confirmed the Big Bang theory, at least until the European Space Agency discovered that dust in the universe emits microwaves of its own that can introduce the same kind of polarization.

"That was a very big deal when it was announced. If [cosmic inflation] had been confirmed, it would have spoken volumes about the nature of our universe, its beginning, its evolution, and its end," said Mezzacappa. "Further analysis, and likely further astronomical data collection, will be required to include the effect of dust."

There are inherent uncertainties in algorithms, models, outcomes, and sometimes the data itself that can impact conclusions. Human nature also plays a part. Here, we explain six of the many reasons the same data can result in different interpretations.

Lisa Morgan is a freelance writer who covers big data and BI for InformationWeek. She has contributed articles, reports, and other types of content to various publications and sites ranging from SD Times to the Economist Intelligent Unit. Frequent areas of coverage include ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 7
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

11 Things IT Professionals Wish They Knew Earlier in Their Careers
Lisa Morgan, Freelance Writer,  4/6/2021
Time to Shift Your Job Search Out of Neutral
Jessica Davis, Senior Editor, Enterprise Apps,  3/31/2021
Does Identity Hinder Hybrid-Cloud and Multi-Cloud Adoption?
Joao-Pierre S. Ruth, Senior Writer,  4/1/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Successful Strategies for Digital Transformation
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll