Big Data // Big Data Analytics
10:37 AM
Connect Directly
Mobile Threats & How to Keep Them at Bay
Jun 01, 2016
With savvy cybercriminals using vulnerabilities in apps, networks, and operating systems to gain c ...Read More>>

5 Data Science Sins To Beware

Repent, ye data scientists! Avoid these five big data evils -- or pay with your immortal soul.

In other words, a lot of different variables could be the cause of something.

A lot of data scientists are under pressure to produce results that favor their employer's or client's hypothesis, Walker pointed out, a situation that can lead to inaccurate, misleading or just plain wrong data analysis.

Sin #3: Data Selection Bias

"This means the skewing of data sources," Walker said. "A lot of times [data scientists] fool themselves in this regard."

How so? By measuring only data that's available. "Oftentimes, what's most valuable or most appropriate for you to be looking at is data that just isn't available yet," said Walker. "And that can really skew the results of the science."

When examining big-data research, it's always important to ask this key question: Who is paying for the data science? "Whoever's paying for it will probably want to skew the data to favor their interests," Walker added.

Sin #4: Narrative Fallacy

"A lot of data scientists feel the need to fit a story into connected or disconnected fact," said Walker. "So they come up with a story, and then they go looking for data that they can plausibly interpret to fit that story."

Real data science doesn't -- or shouldn't -- work that way. So what's the right approach?

"You have a hypothesis, you collect the data, you run experiments … and then you let the chips fall where they may," Walker said. "And you interpret them according to the scientific method, and give [your findings] to the decision makers."

Sin #5: Cognitive Bias

This is where you're skewing data to suit your prior beliefs rather than relying on the evidence.

"This is very dangerous, yet I see it all the time," Walker said. "It's human nature. We all have prior beliefs. We all have biases, even though the best of us try to recognize them and control for it."

In short, data scientists need to focus more on the evidence. "We need to really look at the data to get the facts and the evidence out of it, so that we can make better decisions," he said.

Walker himself 'fesses up to sometimes falling into these data science traps. "I see things I thought were true, and then I see evidence they're not. I was wrong about the way I thought about something," he said. "We need to be humble and look at the evidence. And we need to train people to do that more."

Emerging software tools now make analytics feasible -- and cost-effective -- for most companies. Also in the Brave The Big Data Wave issue of InformationWeek: Have doubts about NoSQL consistency? Meet Kyle Kingsbury's Call Me Maybe project. (Free registration required.)

2 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
2016 InformationWeek Elite 100
Our 28th annual ranking of the leading US users of business technology.
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.