About Us

Advertise

Data Management // Big Data Analytics

7/9/2015
08:06 AM

Lisa Morgan
Slideshows

Connect Directly

6 comments
Comment Now

7 Common Biases That Skew Big Data Results

Flawed data analysis leads to faulty conclusions and bad business outcomes. Beware of these seven types of bias that commonly challenge organizations' ability to make smart decisions.

5 of 8

Simpson's Paradox

A trend that is indicated in groups of data can reverse when the groups of data are combined. This is called Simpson's Paradox. It is one reason medical findings and other types of research first report one thing and then the opposite result at another point in time. It is also one reason why seemingly successful marketing campaigns prove not to be successful after all.

"The most common bias in data analysis is called the Simpson's Paradox," said Rado Kotorov, chief innovation officer at business intelligence and analytics provider Information Builders. "It's important to realize with big data, using descriptive statistics or just data visualization can lead to bias and wrong decisions. The data analysts need to know when to evaluate the trends statistically to determine that the trend is real and also that the factors that contribute to the trend are significant and not random."

A sales and marketing campaign may result in little or no ROI when the customer incentives are based on faulty conclusions. Kotorov tried warning a former employer that more analysis was necessary to validate a trend upon which a marketing campaign was based, but the warning was ignored.

"Instead of driving sales up and increasing the profit, [the campaign] drove margin down and increased the loss. I couldn't convince anyone to take the time to investigate the trend and validate whether it was correct or wrong. When the testing control group measurements came, that's when we had to look at it and see what happened, and we found out that the trend was the wrong trend."

Today's marketers are using marketing analytics tools simultaneously with multivariate testing to slice and dice data which results in different levels of aggregation. However, the averages can be misleading.

"The typical fallacy is if you do things at a fine level of aggregation, but do not find contradictions immediately, then you'll follow your instinct that the trend is a valid trend," Kotorov said.

(Image: Geralt via Pixabay)

5 of 8

Comment |

Email This |

Print |

RSS

Comments

Newest First | Oldest First | Threaded View

[close this box]

50%

LisaMorgan,
User Rank: Moderator
7/25/2015 | 11:23:02 AM

Re: Selection Bias

I thought about including that but it is a subtype of confirmation bias. Also, the cognitive biases are far more familiar to the general population than the others.

Selection bias is an issue as you say.

Reply | Post Message | Messages List | Start a Board

50%

shamika,
User Rank: Ninja
7/24/2015 | 11:17:51 PM

Selection Bias

Well in my opinion I feel selection bias is needed. This is very much important when it comes to population rather than a sample. However there is a concern on accuracy.

Reply | Post Message | Messages List | Start a Board

100%

LisaMorgan,
User Rank: Moderator
7/19/2015 | 1:27:05 PM

Re: Confirmation bias is the big one

When I've written for business audiences only, I've avoided the term, "confirmation bias" and instead endeavored them get them to understand the differences between assumptions and hypotheses. If you assume, you've baked in what you believe to be truth without proof. A hypothesis is tested - proven or disproven. In other words, be prepared to be wrong, embrace that and learn from it. However, is it very common to cherry pick questions, engineer survey questions, and dismiss anything that does not go along with that which one set out to prove.

Confirmation bias isn't always intentional; however. It's sometimes done unconsciously. And that's what people who are genuinely concerned about confirmation bias fear in their own work. Working collaboratively with others, exchanging ideas and comparing results in an open environment is good, ideally if everyone is not out to prove the same point such as Brand X cola with a zillion grams of sugar is a viable form of health food. :-)

Reply | Post Message | Messages List | Start a Board

50%

jries921,
User Rank: Ninja
7/19/2015 | 1:15:47 AM

Re: Confirmation bias is the big one

As Mark Twain pointed out long ago, we all have axes to grind. Given such is the case, the wise/honest thing to do is to recognize one's own biases and try to correct for them. Free and open discussion tends to promote this end, which is why those most wedded to their own ideas or the orthodoxies of their own ingroups will often try to squelch it (unless, of course, they're committed to the concept), or simply withdraw to their own little comfortable groups (caucuses?), so they're not subject to the discomfort of cognitive dissonance. And if partisan or other factional politics start playing a prominent role, it can be very difficult to reach any sort of reasonable consensus (witness what has happened to macroeconomics since the 1980s; or constitutional law since forever).

Another issue is that those most wedded to their ideas are often the very people most motivated to see where they lead. This can actually be a good thing as the necessary research will often be very hard work the "impartial inquirer after Truth" (unless he is particularly dilligent) might well seek to avoid. Would Charles Darwin have chosen evolutionary biology as his field of research if his grandfather had not been an early proponent of the concept? I suspect not. If such people can resist the urge to cheat, or perhaps can be persuaded to collaborate with those much more skeptical so they can "keep each other honest", then they can do very good work indeed.

it is natural for scientific journals and their readers to be much more interested in successes than failures (I don't expect that to ever change), and pride makes it difficult to let go of one's life's work when it reaches a dead end (perhaps this is part of why so many breakthroughs are made by younger researchers); but even experimental failures will often lead to discovery if one is willing to consider the implications (thus, failure can lead to success). A classic example was the "failed" 1887 Michelson–Morley experiment that was a large part of the experimental basis for Einstein's Special Theory of Relativity.

Reply | Post Message | Messages List | Start a Board

100%

LisaMorgan,
User Rank: Moderator
7/18/2015 | 7:14:16 PM

Re: Confirmation bias is the big one

Confirmation bias is everywhere. Sometime it's deliberate, sometimes it isn't. A data scientists and a machine learning specialist have been trying to get me to talk about the issue in terms of scientific journals because scientific journals only publish positive results, they say. When it comes to healthcare, confirmation bias can be very dangerous.

Reply | Post Message | Messages List | Start a Board

50%

jries921,
User Rank: Ninja
7/18/2015 | 3:50:07 PM

Confirmation bias is the big one

I've long thought that the most effective propaganda is that which tells people what they already think; which is why partisan talk radio excels at radicalizing those who already believe, but does little or nothing to persuade people who don't (if anything, the "shock jock" commentary turns them off). Confirmation bias is also a very good reason to be skeptical of any research sponsored by a for-profit corporation; as even if the researchers and sponsors are working in good faith (a dubious proposition in an era in which profit maximization is widely thought to trump all other considerations), vendors will tend to disbelieve any research that makes their products look bad (few people want to believe they're selling junk).

Reply | Post Message | Messages List | Start a Board