Software // Information Management
Commentary
6/20/2007
07:51 AM
Neil Raden
Neil Raden
Commentary
50%
50%

People Matter in Advanced Analytics

I read with interest Tom Davenport's article, "Humans and Black Boxes" in BIReview. He raises the issue about whether humans are required in the analytics process anymore, given the offerings of vendors of unattended data mining tools. After all, with all of the hardware and bandwidth at our disposal, shouldn't systems be smart enough to swim around in the data and come up with predictive models that are more accurate than we mere humans can? Of course, Davenport doesn't believe that, and neithe

I read with interest Tom Davenport's article, "Humans and Black Boxes" in the June, 2007 issue of BIReview. He raises the issue about whether humans are required in the analytics process anymore, given the offerings of vendors of unattended data mining tools. After all, with all of the hardware and bandwidth at our disposal, shouldn't systems be smart enough yet to swim around in the data and come up with predictive models that are more accurate than we mere humans can? Of course, Davenport doesn't believe that, and neither do I.Tom's earlier article in the Harvard Business Review, "Competing on Analytics" (and subsequent book of the same name), explored the necessity of using advanced analytics as a competitive weapon. He argued that traditional means of competition, such as price, customer loyalty or speed to market, among others, were still necessary but not sufficient. Having an analytical culture, straight up to the CEO's office was the ticket, supported by a cadre of PhD's who really knew how to grind the numbers. We disagreed a little on this point, but on the major point, that analytics are now on the critical path, there is no disagreement. James Taylor and I wrote about Enterprise Decision Management, which is designed to automate many kinds of decisions, but there are people in the process from top to bottom. Davenport does try to segment analytics into three kinds of activities - human-centered, black box and visualization. The human-centered type is the "quant jock" who uses SAS or SPSS and does everything more or less by hand. In contrast, the black box type is completely machine oriented. I think there is a middle ground, which is a combination of quant's and domain experts working together using both coding tools like SAS and advanced analytical tools. In my experience, this is the combination that always has the best result, not necessarily optimal statistical result, but a good one that has a chance to get implemented. Most clients I've worked for will not implement a meaningful change in the way they operate without some degree of socialization of the issue. That requires explaining, through narrative, metaphor and analogy, things computers cannot do. Any useful analytics initiative should have as its primary goals, not only predicting the near future, but also explaining how the model works and providing a continuous, evidence-based refining process. Computers are good at certain things, like following instructions and crunching numbers. Because of that they can rapidly find associations, dependencies and anomalies that humans miss. Humans are better at explaining, communicating and, most importantly of all, discerning what really needs attention. Tom cautions that it is wise to take inventory of your human capital before your embark on an analytics initiative. You'll need all you can get.

Neil Raden is the founder of Hired Brains, providers of consulting, research and analysis in Business Intelligence, Performance Management, real-time analytics and information/semantic integration. Neil is co-author of the just-released book "Smart Enough Systems," with business rules expert James Taylor.I read with interest Tom Davenport's article, "Humans and Black Boxes" in BIReview. He raises the issue about whether humans are required in the analytics process anymore, given the offerings of vendors of unattended data mining tools. After all, with all of the hardware and bandwidth at our disposal, shouldn't systems be smart enough to swim around in the data and come up with predictive models that are more accurate than we mere humans can? Of course, Davenport doesn't believe that, and neither do I.

Comment  | 
Print  | 
More Insights
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest, Dec. 9, 2014
Apps will make or break the tablet as a work device, but don't shortchange critical factors related to hardware, security, peripherals, and integration.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of December 7, 2014. Be here for the show and for the incredible Friday Afternoon Conversation that runs beside the program!
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.