5 Data Science Sins To Beware - InformationWeek
Data Management // Big Data Analytics
10:37 AM
Connect Directly
Out of the Black Box: Selling Security to your C-suite
Jul 20, 2017
To maximize the return on cloud security investments, CISOs need a seat at the table. Unfortunatel ...Read More>>

5 Data Science Sins To Beware

Repent, ye data scientists! Avoid these five big data evils -- or pay with your immortal soul.

In other words, a lot of different variables could be the cause of something.

A lot of data scientists are under pressure to produce results that favor their employer's or client's hypothesis, Walker pointed out, a situation that can lead to inaccurate, misleading or just plain wrong data analysis.

Sin #3: Data Selection Bias

"This means the skewing of data sources," Walker said. "A lot of times [data scientists] fool themselves in this regard."

How so? By measuring only data that's available. "Oftentimes, what's most valuable or most appropriate for you to be looking at is data that just isn't available yet," said Walker. "And that can really skew the results of the science."

When examining big-data research, it's always important to ask this key question: Who is paying for the data science? "Whoever's paying for it will probably want to skew the data to favor their interests," Walker added.

Sin #4: Narrative Fallacy

"A lot of data scientists feel the need to fit a story into connected or disconnected fact," said Walker. "So they come up with a story, and then they go looking for data that they can plausibly interpret to fit that story."

Real data science doesn't -- or shouldn't -- work that way. So what's the right approach?

"You have a hypothesis, you collect the data, you run experiments … and then you let the chips fall where they may," Walker said. "And you interpret them according to the scientific method, and give [your findings] to the decision makers."

Sin #5: Cognitive Bias

This is where you're skewing data to suit your prior beliefs rather than relying on the evidence.

"This is very dangerous, yet I see it all the time," Walker said. "It's human nature. We all have prior beliefs. We all have biases, even though the best of us try to recognize them and control for it."

In short, data scientists need to focus more on the evidence. "We need to really look at the data to get the facts and the evidence out of it, so that we can make better decisions," he said.

Walker himself 'fesses up to sometimes falling into these data science traps. "I see things I thought were true, and then I see evidence they're not. I was wrong about the way I thought about something," he said. "We need to be humble and look at the evidence. And we need to train people to do that more."

Emerging software tools now make analytics feasible -- and cost-effective -- for most companies. Also in the Brave The Big Data Wave issue of InformationWeek: Have doubts about NoSQL consistency? Meet Kyle Kingsbury's Call Me Maybe project. (Free registration required.)

2 of 2
Comment  | 
Print  | 
More Insights
Threaded  |  Newest First  |  Oldest First
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
IT Strategies to Conquer the Cloud
Chances are your organization is adopting cloud computing in one way or another -- or in multiple ways. Understanding the skills you need and how cloud affects IT operations and networking will help you adapt.
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of November 6, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll