7 Common Biases That Skew Big Data Results - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Big Data Analytics
News
7/9/2015
08:06 AM
Lisa Morgan
Lisa Morgan
Slideshows
Connect Directly
Twitter
RSS
E-Mail

7 Common Biases That Skew Big Data Results

Flawed data analysis leads to faulty conclusions and bad business outcomes. Beware of these seven types of bias that commonly challenge organizations' ability to make smart decisions.
2 of 8

Confirmation Bias 

Confirmation bias occurs when there's an intentional or unintentional desire to prove a hypothesis, assumption, or opinion. It is one of the most common types of cognitive bias -- and one that humans easily fall victim to -- because the data often 'feels right.'
'You'd think producing a statistic is an objective process, but actually there's a lot of flexibility that goes into a process in terms of what variables you calculate and how you calculate them,' said ClearerThinking.org founder Spencer Greenberg. 'This happens a lot [when] people have an idea of what should be done or [as] a bad result of analysis. They end up finding a way to get the statistic to agree with [their original idea] and rationalize it afterwards.'
Patrick Rice, CEO of predictive analytics startup Lumidatum, and others interviewed for this article, agree that there is a very real danger of business leaders transferring bias to data scientists and analysts. Quite often, business leaders treat the data team like the business intelligence team, meaning their approach to collaboration is, 'Go build this report.' However, in the case of confirmation bias, the directive is more along the lines of, 'Make sure the data agrees with my point of view.'
'[Confirmation bias transfer] can be especially dangerous given that the analysis in these scenarios is often used to make important business decisions,' said Lumidatum's Rice. '[It] can lead to some pretty damaging results.' 
(Image: nieverlaan via Pixabay)

Confirmation Bias

Confirmation bias occurs when there's an intentional or unintentional desire to prove a hypothesis, assumption, or opinion. It is one of the most common types of cognitive bias -- and one that humans easily fall victim to -- because the data often "feels right."

"You'd think producing a statistic is an objective process, but actually there's a lot of flexibility that goes into a process in terms of what variables you calculate and how you calculate them," said ClearerThinking.org founder Spencer Greenberg. "This happens a lot [when] people have an idea of what should be done or [as] a bad result of analysis. They end up finding a way to get the statistic to agree with [their original idea] and rationalize it afterwards."

Patrick Rice, CEO of predictive analytics startup Lumidatum, and others interviewed for this article, agree that there is a very real danger of business leaders transferring bias to data scientists and analysts. Quite often, business leaders treat the data team like the business intelligence team, meaning their approach to collaboration is, "Go build this report." However, in the case of confirmation bias, the directive is more along the lines of, "Make sure the data agrees with my point of view."

"[Confirmation bias transfer] can be especially dangerous given that the analysis in these scenarios is often used to make important business decisions," said Lumidatum's Rice. "[It] can lead to some pretty damaging results."

(Image: nieverlaan via Pixabay)

2 of 8
Comment  | 
Print  | 
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

Slideshows
11 Things IT Professionals Wish They Knew Earlier in Their Careers
Lisa Morgan, Freelance Writer,  4/6/2021
News
Time to Shift Your Job Search Out of Neutral
Jessica Davis, Senior Editor, Enterprise Apps,  3/31/2021
Commentary
Does Identity Hinder Hybrid-Cloud and Multi-Cloud Adoption?
Joao-Pierre S. Ruth, Senior Writer,  4/1/2021
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Successful Strategies for Digital Transformation
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Slideshows
Flash Poll