Data Outliers: 10 Ways To Prevent Big Data Damage - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Big Data Analytics
News
4/18/2016
07:06 AM
Lisa Morgan
Lisa Morgan
Slideshows
Connect Directly
Twitter
RSS
E-Mail

Data Outliers: 10 Ways To Prevent Big Data Damage

Most business decision-makers aren't trained to understand data outliers, but they can learn the basics. Executives, managers, and employees without math degrees can ask smarter questions about analyses they're basing crucial judgments on. Here are some things to know.
10 of 11

Beware Of Assumptions  

As with most things, assumptions about data can be misleading, and biases can impact the outcome of an analysis.  
'Sampling mistakes and wrong assumptions about underlying statistical distributions are typical mistakes people make. It's also common to see people use statistical tests or analytical packages without understanding what the underlying assumptions are,' said Parvez Ahammad, head of the data science and machine learning group at application delivery platform provider Instart Logic, in an interview.  
'Check your prior assumptions or beliefs about the data and make sure they are valid, be open-minded about what the data tells you, collect as much data as you can so you have a good enough sample set to arrive at the decision, and, if you encounter outliers, check with other folks who may have the expertise [to understand them] and offer an alternative explanation.'
(Image: cor125 via Pixabay)

Beware Of Assumptions

As with most things, assumptions about data can be misleading, and biases can impact the outcome of an analysis.

"Sampling mistakes and wrong assumptions about underlying statistical distributions are typical mistakes people make. It's also common to see people use statistical tests or analytical packages without understanding what the underlying assumptions are," said Parvez Ahammad, head of the data science and machine learning group at application delivery platform provider Instart Logic, in an interview.

"Check your prior assumptions or beliefs about the data and make sure they are valid, be open-minded about what the data tells you, collect as much data as you can so you have a good enough sample set to arrive at the decision, and, if you encounter outliers, check with other folks who may have the expertise [to understand them] and offer an alternative explanation."

(Image: cor125 via Pixabay)

10 of 11
Comment  | 
Print  | 
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

Commentary
Why IT Leaders Should Make Cloud Training a Top Priority
John Edwards, Technology Journalist & Author,  4/14/2021
Slideshows
10 Things Your Artificial Intelligence Initiative Needs to Succeed
Lisa Morgan, Freelance Writer,  4/20/2021
Commentary
Lessons I've Learned From My Career in Technology
Guest Commentary, Guest Commentary,  5/4/2021
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Slideshows
Flash Poll