Repent, ye data scientists! Avoid these five big data evils -- or pay with your immortal soul.
In other words, a lot of different variables could be the cause of something.
A lot of data scientists are under pressure to produce results that favor their employer's or client's hypothesis, Walker pointed out, a situation that can lead to inaccurate, misleading or just plain wrong data analysis.
Sin #3: Data Selection Bias
"This means the skewing of data sources," Walker said. "A lot of times [data scientists] fool themselves in this regard."
How so? By measuring only data that's available. "Oftentimes, what's most valuable or most appropriate for you to be looking at is data that just isn't available yet," said Walker. "And that can really skew the results of the science."
When examining big-data research, it's always important to ask this key question: Who is paying for the data science? "Whoever's paying for it will probably want to skew the data to favor their interests," Walker added.
Sin #4: Narrative Fallacy
"A lot of data scientists feel the need to fit a story into connected or disconnected fact," said Walker. "So they come up with a story, and then they go looking for data that they can plausibly interpret to fit that story."
Real data science doesn't -- or shouldn't -- work that way. So what's the right approach?
"You have a hypothesis, you collect the data, you run experiments … and then you let the chips fall where they may," Walker said. "And you interpret them according to the scientific method, and give [your findings] to the decision makers."
Sin #5: Cognitive Bias
This is where you're skewing data to suit your prior beliefs rather than relying on the evidence.
"This is very dangerous, yet I see it all the time," Walker said. "It's human nature. We all have prior beliefs. We all have biases, even though the best of us try to recognize them and control for it."
In short, data scientists need to focus more on the evidence. "We need to really look at the data to get the facts and the evidence out of it, so that we can make better decisions," he said.
Walker himself 'fesses up to sometimes falling into these data science traps. "I see things I thought were true, and then I see evidence they're not. I was wrong about the way I thought about something," he said. "We need to be humble and look at the evidence. And we need to train people to do that more."
Emerging software tools now make analytics feasible -- and cost-effective -- for most companies. Also in the Brave The Big Data Wave issue of InformationWeek: Have doubts about NoSQL consistency? Meet Kyle Kingsbury's Call Me Maybe project. (Free registration required.)
6 Tools to Protect Big DataMost IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Big Data Brings Big Security ProblemsWhy should big data be more difficult to secure? In a word, variety. But the business won’t wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more.
InformationWeek Tech Digest, Nov. 10, 2014Just 30% of respondents to our new survey say their companies are very or extremely effective at identifying critical data and analyzing it to make decisions, down from 42% in 2013. What gives?