7 Common Biases That Skew Big Data Results - InformationWeek
IoT
IoT
Data Management // Big Data Analytics
News
7/9/2015
08:06 AM
Lisa Morgan
Lisa Morgan
Slideshows
100%
0%

7 Common Biases That Skew Big Data Results

Flawed data analysis leads to faulty conclusions and bad business outcomes. Beware of these seven types of bias that commonly challenge organizations' ability to make smart decisions.
Previous
1 of 8
Next

(Image: Geralt via Pixabay)

(Image: Geralt via Pixabay)

Data-driven decision-making is considered a smart move, but it can be costly or dangerous when something that appears to be true is not actually true. Even with the best of intentions, some of the world's most famous companies are challenged by skewed results because the data is biased, or the humans collecting and analyzing data are biased, or both.

"There are a lot of perils of data analysis. Some people say, 'The number says X, therefore we should do X,' but there is more nuance, [such as] how the number was calculated and whether you are able to make the inference from the number you think you can make," said Spencer Greenberg, a mathematician and founder of decision-making-tool provider ClearerThinking.org, in an interview.

Various forms of bias seem rather obvious in marketing and political campaigns; however, bias is not always obvious. Some types can be difficult to identify. Neither are biases always mutually exclusive, meaning that several biases may arise in a single use-case. Because bias is pervasive and not always obvious, business leaders and members of any data team should be aware of it and take steps to avoid it (or at least minimize the effect).

Data teams are familiar with different kinds of bias and their effects on analytical results and conclusions. Business leaders are less likely to know the details of specific bias types, but they likely have experienced the effects of bias firsthand when a project or initiative did not yield the expected results.

"Business leaders can ask data scientists for something as simple as a confusion matrix -- how many false positives are we seeing, how many false negatives are we seeing, and how many members of the population fall into the four boxes [true positive, true negative, false positive, and false negative]," said Keith Schleicher, managing director of decision science at Internet service provider CenturyLink in an interview. "They can also ask what kind of sampling was done and whether the sample size is accurate."

While such terminology has not been used much at the business level, it is becoming more common because business leaders are responsible for the data-based decisions they make and should question things like the quality of data, where it comes from, how it was gathered, and what's behind the numbers.

Here we present seven types of cognitive and data bias that commonly challenge organizations' decision-making. Once you've reviewed these, tell us in the comments section below whether you've experienced any in your organization, and how that worked out for you.

Lisa Morgan is a freelance writer who covers big data and BI for InformationWeek. She has contributed articles, reports, and other types of content to various publications and sites ranging from SD Times to the Economist Intelligent Unit. Frequent areas of coverage include ... View Full Bio

Previous
1 of 8
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
LisaMorgan
50%
50%
LisaMorgan,
User Rank: Moderator
7/25/2015 | 11:23:02 AM
Re: Selection Bias
I thought about including that but it is a subtype of confirmation bias.  Also, the cognitive biases are far more familiar to the general population than the others.

Selection bias is an issue as you say.
shamika
50%
50%
shamika,
User Rank: Ninja
7/24/2015 | 11:17:51 PM
Selection Bias
Well in my opinion I feel selection bias is needed.  This is very much important when it comes to population rather than a sample. However there is a concern on accuracy.
LisaMorgan
100%
0%
LisaMorgan,
User Rank: Moderator
7/19/2015 | 1:27:05 PM
Re: Confirmation bias is the big one
When I've written for business audiences only, I've avoided the term, "confirmation bias" and instead endeavored them get them to understand the differences between assumptions and hypotheses.  If you assume, you've baked in what you believe to be truth without proof.  A hypothesis is tested - proven or disproven.  In other words, be prepared to be wrong, embrace that and learn from it.  However, is it very common to cherry pick questions, engineer survey questions, and dismiss anything that does not go along with that which one set out to prove.  

Confirmation bias isn't always intentional; however.  It's sometimes done unconsciously.  And that's what people who are genuinely concerned about confirmation bias fear in their own work.  Working collaboratively with others, exchanging ideas and comparing results in an open environment is good, ideally if everyone is not out to prove the same point such as Brand X cola with a zillion grams of sugar is a viable form of health food.  :-)

 
jries921
50%
50%
jries921,
User Rank: Ninja
7/19/2015 | 1:15:47 AM
Re: Confirmation bias is the big one
As Mark Twain pointed out long ago, we all have axes to grind.  Given such is the case, the wise/honest thing to do is to recognize one's own biases and try to correct for them.  Free and open discussion tends to promote this end, which is why those most wedded to their own ideas or the orthodoxies of their own ingroups will often try to squelch it (unless, of course, they're committed to the concept), or simply withdraw to their own little comfortable groups (caucuses?), so they're not subject to the discomfort of cognitive dissonance.  And if partisan or other factional politics start playing a prominent role, it can be very difficult to reach any sort of reasonable consensus (witness what has happened to macroeconomics since the 1980s; or constitutional law since forever).

Another issue is that those most wedded to their ideas are often the very people most motivated to see where they lead.  This can actually be a good thing as the necessary research will often be very hard work the "impartial inquirer after Truth" (unless he is particularly dilligent) might well seek to avoid.  Would Charles Darwin have chosen evolutionary biology as his field of research if his grandfather had not been an early proponent of the concept?  I suspect not.  If such people can resist the urge to cheat, or perhaps can be persuaded to collaborate with those much more skeptical so they can "keep each other honest", then they can do very good work indeed.

it is natural for scientific journals and their readers to be much more interested in successes than failures (I don't expect that to ever change), and pride makes it difficult to let go of one's life's work when it reaches a dead end (perhaps this is part of why so many breakthroughs are made by younger researchers); but even experimental failures will often lead to discovery if one is willing to consider the implications (thus, failure can lead to success).  A classic example was the "failed" 1887 Michelson–Morley experiment that was a large part of the experimental basis for Einstein's Special Theory of Relativity.
LisaMorgan
100%
0%
LisaMorgan,
User Rank: Moderator
7/18/2015 | 7:14:16 PM
Re: Confirmation bias is the big one
Confirmation bias is everywhere.  Sometime it's deliberate, sometimes it isn't.  A data scientists and a machine learning specialist have been trying to get me to talk about the issue in terms of  scientific journals because scientific journals only publish positive results, they say.  When it comes to healthcare, confirmation bias can be very dangerous.  
jries921
50%
50%
jries921,
User Rank: Ninja
7/18/2015 | 3:50:07 PM
Confirmation bias is the big one
I've long thought that the most effective propaganda is that which tells people what they already think; which is why partisan talk radio excels at radicalizing those who already believe, but does little or nothing to persuade people who don't (if anything, the "shock jock" commentary turns them off).  Confirmation bias is also a very good reason to be skeptical of any research sponsored by a for-profit corporation; as even if the researchers and sponsors are working in good faith (a dubious proposition in an era in which profit maximization is widely thought to trump all other considerations), vendors will tend to disbelieve any research that makes their products look bad (few people want to believe they're selling junk).

 
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
Digital Transformation Myths & Truths
Transformation is on every IT organization's to-do list, but effectively transforming IT means a major shift in technology as well as business models and culture. In this IT Trend Report, we examine some of the misconceptions of digital transformation and look at steps you can take to succeed technically and culturally.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll