7 Common Biases That Skew Big Data Results - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Big Data Analytics
News
7/9/2015
08:06 AM
Lisa Morgan
Lisa Morgan
Slideshows
Connect Directly
Twitter
RSS
E-Mail

7 Common Biases That Skew Big Data Results

Flawed data analysis leads to faulty conclusions and bad business outcomes. Beware of these seven types of bias that commonly challenge organizations' ability to make smart decisions.
7 of 8

Confounding Variables 
Sometimes a perceived relationship between two variables may be proven partially false or entirely false because a confounding variable has been omitted (often because it has been overlooked).  
'It could be that different populations are collected or reported differently or by different people, a causal variable that affects the behavior of each population, or an inherent quality that leads to autocorrelation,' said Metrocosm's Max Galka. 
Schleicher once worked on a survey that asked respondents which credit card brands they would consider. Over a three-year period, the data indicated that the consideration numbers for one credit card company nearly doubled, while those of several other companies remained flat. The obvious conclusion turned out to be the wrong conclusion.
'A confounding variable is cardholders have higher consideration for their current credit card companies than people who are not customers,' said CenturyLink's Schleicher. 'The company had gone through several M&As, and their portfolio had grown enormously over that three-year period. They hadn't improved their consideration, or customer experience, or how their customers valued them. They just acquired more customers through portfolio mergers. 
'If you exploded it out by customers and non-customers, you'd see that non-customer considerations were flat and the only thing that had changed was their marketshare. It's easy to look at a couple of variables together. You're looking at correlations, you plot things, you see a pattern that looks promising, and you have to ask yourself whether it's the relationship or something else explaining it.'
(Image: ChadoNihi via Pixabay)

Confounding Variables

Sometimes a perceived relationship between two variables may be proven partially false or entirely false because a confounding variable has been omitted (often because it has been overlooked).

"It could be that different populations are collected or reported differently or by different people, a causal variable that affects the behavior of each population, or an inherent quality that leads to autocorrelation," said Metrocosm's Max Galka.

Schleicher once worked on a survey that asked respondents which credit card brands they would consider. Over a three-year period, the data indicated that the consideration numbers for one credit card company nearly doubled, while those of several other companies remained flat. The obvious conclusion turned out to be the wrong conclusion.

"A confounding variable is cardholders have higher consideration for their current credit card companies than people who are not customers," said CenturyLink's Schleicher. "The company had gone through several M&As, and their portfolio had grown enormously over that three-year period. They hadn't improved their consideration, or customer experience, or how their customers valued them. They just acquired more customers through portfolio mergers.

"If you exploded it out by customers and non-customers, you'd see that non-customer considerations were flat and the only thing that had changed was their marketshare. It's easy to look at a couple of variables together. You're looking at correlations, you plot things, you see a pattern that looks promising, and you have to ask yourself whether it's the relationship or something else explaining it."

(Image: ChadoNihi via Pixabay)

7 of 8
Comment  | 
Print  | 
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

Slideshows
11 Things IT Professionals Wish They Knew Earlier in Their Careers
Lisa Morgan, Freelance Writer,  4/6/2021
News
Time to Shift Your Job Search Out of Neutral
Jessica Davis, Senior Editor, Enterprise Apps,  3/31/2021
Commentary
Does Identity Hinder Hybrid-Cloud and Multi-Cloud Adoption?
Joao-Pierre S. Ruth, Senior Writer,  4/1/2021
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Successful Strategies for Digital Transformation
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Slideshows
Flash Poll