Nate Silver's Big Data Lessons For The Enterprise - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

CIO Insights & Innovation
Security & Risk Strategy
Team Building & Staffing
IT Strategy
Digital Business
Project Management
Programming Languages
Dr. Dobb's
Enterprise Applications
Operating Systems
Productivity/Collaboration Apps
Network Security
Careers & People
Threat Intelligence
IoT
Attacks & Breaches
Application Security
Cloud Security
Endpoint Security
Mobile Security
Perimeter Security
Risk Management
Operations Security
Analytics
Vulnerabilities & Threats
Security & Risk Strategy
Infrastructure as a Service
Platform as a Service
Software as a Service
Cloud Storage
Data Centers
Mobile Applications
Mobile Devices
Mobile Business
Enterprise Mobility Management
AI/Machine Learning
Big Data Analytics
Hardware/Architectures
Software Platforms
IoT
Networking
Wireless Infrastructure
Data Centers
Cloud Infrastructure
Careers and Certifications
Network Security
Government
Healthcare
Wall Street & Technology
Bank Systems & Technology
Insurance & Technology
Industries
Government
Healthcare
Wall Street & Technology
Bank Systems & Technology
Insurance & Technology
IT Life
Careers
Mobile
Mobile Applications
Mobile Devices
Mobile Business
Enterprise Mobility Management
Software
Enterprise Applications
Operating Systems
Productivity/Collaboration Apps
Comments
Nate Silver's Big Data Lessons For The Enterprise
Newest First  |  Oldest First  |  Threaded View
Page 1 / 2   >   >>
Cindi Howson, BIScorecard
50%
50%
Cindi Howson, BIScorecard,
 User Rank: Apprentice
11/12/2012 | 3:12:51 PM
re: Nate Silver's Big Data Lessons For The Enterprise
Great article, Eric.

That Silver got it right while many political pundits did not does remind me of Moneyball. Your points about "rigor, innovation and looking outside your business confines to find inspiration" are important. So much about BI has been about the basics of data access and looking at numbers the same way. I suspect the next phase of business success will be based on those willing to look at data in new ways ... and sometimes new data.
Regards,
Cindi
MedicalQuack
50%
50%
MedicalQuack,
 User Rank: Moderator
11/12/2012 | 5:12:44 AM
re: Nate Silver's Big Data Lessons For The Enterprise
Let's tie in a bit of luck here too. He does admit and rightly so to the huge error factors in polls and some of predicting is keeping up on credible news...and sometimes we don't get enough of that. We are a naive society with Algorithms and Formulas and I've been writing about for around 3 years now I do research a lot of this in healthcare, plus I used to write code which is very closely related to math.

Actually on Wall Street the Quants write the formulas and the technicians do the code work after words to query and sometime bury algorithms in code modules as software is nothing but a bunch of algorithms working together, in the words of Bill Gates. We get "Algo Duped" as I call it and have had a few professors agree along with folks from the National Institute of Statistical Science that keep telling me to keep making noise.

Here's a series of 4 videos at the link below done folks smarter than me and if you watch all 4 you will soon see that behavioral predictive analytics is going to end up being one huge are for fraud and again gaining money, algorithms move money and they are not very stable. It is what it is. Take the case in Italy where the scientists were guilty of manslaughter for not predicting the earthquakes, naive public put way too much value in their mathematics and formulas as we are not there...yet..and who's to say that someday it could get closer but not for a long time on that item. I live in southern California and believe me if someone could have figured out how to sue Cal Tech for not predicting an earthquake, it would have been done here with all the smart folks that live here <grin>.

I see it all the time and 3 years ago I wrote an article asking if the US needs a Department of Algorithms, nobody checks the math and just like there are 50 ways to leave your lover there's at least that if not more than 50 ways to snow the public:)

Here's Algo Duping 101...

http://www.ducknet.net/attack-...

On a related topic, and some of the same videos are also here, I am up to 46 chapters now on the Attack of the Killer Algorithms as well. In healthcare, United is a good example of using code and mathematics to short pay doctors and hospitals going back 15 years, long time and the formulas worked well for them and made tons of money. They worked so well that Blue Cross, Health Net and the other big guys licensed the customary fee data base for their claim payments...it's been around a long time but is now accelerated.

Again, predictive analytics if not regulated and checked for accurate code and formulas stands to be one of the largest technology rips on the American consumer for sure.

http://ducknetweb.blogspot.com...

I might also add I was not really happy with the Nobel prize for the algorithms used for the medical students finding where they should serve their internship either. It's the same as what's used for dating sites so I didn't see any awards for Match.com or Plenty of Fish handed out:) What he did was a good thing of course but there were better choices I felt instead of just throwing an award out for an algorithm as programmers write them day in and day out and ones of bigger value than that one. The Clinton Foundation or the Clinton's themselves, a much better choice as it also adds some humanism along the line, which we need with everyone getting Algo Duped which leads to some very skewed perceived realities when they are really not there. </grin>
Canamjay
50%
50%
Canamjay,
 User Rank: Guru
11/10/2012 | 7:36:45 PM
re: Nate Silver's Big Data Lessons For The Enterprise
Great article and telling on so many levels. One of my great peeves is the totally predictable phrase always accompanying any public announcement of virtually any monitored stat today: 'surprising economists', 'confounding economic watch-dogs',
'much to the surprise of experts'... etcetera. These folks with Ph.D.'s , one would assume constantly monitored models, access to all the data and computer power they want/need, ALWAYS register 'surprise' when unemployment, GDP, CPI, various market performance indexes, don't match their predictions. This is always much more discouraging to me than the figures. Because, these are the 'experts' advising leadership on actions to effect results. As a society, I often feel we are vastly over-educated and woefully under skilled. Nate simply shows us: the tools are available, we just fail to use them correctly. What a revelation. Thanks for this.
Patrick Taylor
50%
50%
Patrick Taylor,
 User Rank: Apprentice
11/10/2012 | 6:23:12 PM
re: Nate Silver's Big Data Lessons For The Enterprise
It's all about practically using analysis to all of the detailed information to make smarter decisions - whether calling a presidential election or winning baseball games. As pointed out in he article the key is "develop the best decision at the right time and deliver the information to the people that most need the information."

We've found that an important dimension for leveraging analytics is putting them to work at the frontline of business. In the Moneyball story Oakland goes on the winning streak when the players use the statistical insights to play the game smarter - you should take the walk in the 8th inning.

The thing we have to recognize in business is that it's more than executives making smarter decisions, it's everyone making smarter decisions by analyzing all the information we have available.
amywohl
50%
50%
amywohl,
 User Rank: Apprentice
11/9/2012 | 6:38:34 PM
re: Nate Silver's Big Data Lessons For The Enterprise
Excellent article. I agree that knowing your business and taking a customer point of view is much more important than exactly following Nate Silver's process.
jries921
50%
50%
jries921,
 User Rank: Ninja
11/9/2012 | 5:32:14 PM
re: Nate Silver's Big Data Lessons For The Enterprise
I'm starting to gain great appreciation for the policy adopted by Hari Seldon and his followers in Isaac Azimov's "Foundation" series of being very careful about what predictions they made when to keep people from reacting to them. It appears that Nate Silver did some great work, but I wonder if it might be so good that if it gets too much publicity it might affect future election results.
Magzilla
50%
50%
Magzilla,
 User Rank: Apprentice
11/9/2012 | 2:11:29 PM
re: Nate Silver's Big Data Lessons For The Enterprise
Great article! We see creativity entering the picture of wrangling big data early on as a way to form a hypothesis, like most science experiments. And like all science experiments, data analytics are seeking out facts. Businesses ignoring this intelligence and basing decisions off of "gut" reactions will soon fall behind. Here is another article discussing this: < a href= " http://themoreyouknowbandb.wor..." _blank " > The More You Know < /a >
cbabcock
50%
50%
cbabcock,
 User Rank: Strategist
11/8/2012 | 9:30:21 PM
re: Nate Silver's Big Data Lessons For The Enterprise
Silver's willingness to examine polling data in individual states and compare it to national polls gave him great insight into how the swing states might go. Outcomes in Iowa, Fla., Ohio, Virginia are mercurial, sometimes red, sometimes blue, but can be predicted for a given election if you've established a past relationship between those two sets of data. I think he knows how to spot trends in the national data, then winnows it among eight or nine swing states--how much will the trend apply in this setting? Hard to believe but Silver was the topic of discussion following our AARP-league basketball game last night. Charlie Babcock
EricLundquist
50%
50%
EricLundquist,
 User Rank: Apprentice
11/8/2012 | 8:08:10 PM
re: Nate Silver's Big Data Lessons For The Enterprise
Thanks for the comment. I don't think this sounds ike an ad for Big Data, but you are entitled to your opinion. I think it is clear that I said Silver's success is not about big data but about an innovative use of data big and small.
Ellis Booker
50%
50%
Ellis Booker,
 User Rank: Moderator
11/8/2012 | 5:49:09 PM
re: Nate Silver's Big Data Lessons For The Enterprise
This bit right here should be printed out and hung up at every water cooler and coffee machine in corporate America: "There is no magic to Silver's methods. There is hard work, a willingness to make mistakes and adjust, and a realization that the common wisdom is sometimes not wisdom at all." --Ellis Booker, InformationWeek Community Editor.
Page 1 / 2   >   >>


State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Download Now!
Editors' Choice
Slideshows
IT Careers: Top 10 US Cities for Tech Jobs
Cynthia Harvey, Freelance Journalist, InformationWeek,  1/14/2020
Commentary
Predictions for Cloud Computing in 2020
James Kobielus, Research Director, Futurum,  1/9/2020
News
What's Next: AI and Data Trends for 2020 and Beyond
Jessica Davis, Senior Editor, Enterprise Apps,  12/30/2019
Register for InformationWeek Newsletters
Video
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Download This Issue!
White Papers
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Sponsored Video
Flash Poll