Models Take the Danger Out of Prediction

Cindi Howson asked in a recent blog, "Given the perceived value of predictive analytics, why does it seem to have had such lack luster success to date? Like most things, I suspect the answer is part cultural and part technological." I couldn't agree more. Predictive modeling isn't a crystal ball, and despite the efforts of Business Objects, Hyperion, Microstrategy and SAS to get predictive modeling into mainstream BI tools, there are a lot of other reasons for its lack of success. Knowledge Discovery is something that is best left to the experts, those with PhDs, years of experience or both.It may be easy to navigate in the some of the newer tools and poke around with a Kohonen Network or some association algorithms, but unless you understand the whole process, the likelihood of producing reasonable looking but spurious results is pretty high. BI's sort of singular, interactive and casual metaphor is very different from the practice of Knowledge Discovery, which involves a deep understanding of the mathematics of data mining, of training set design and interpretation of the results. Predictive modeling is different from using predictive models. In the latter case, a rule may be put in place that no one with a FICO score under 600 will be offered financing for a new Prius because the likelihood of default is unacceptably high, or that someone purchasing a book about A.J. Ayer would be interested in an offer for a corduroy jacket with patches on the sleeves?. But where did the FICO score come from in the first place and where did the knowledge that scores under 600 tend to default? What led to the association between Logical Positivism and retro/academic fashion? That is predictive modeling and is not within the realm of end-user, BI computing. Automating decisions in organizations based on the use of predictive models is a good idea, especially when those decisions are numerous and (each one) carries fairly low risk for error. Allowing non-professionals to apply Knowledge Discovery techniques and act on them is dangerous. As a result, using predictive modeling that is developed by a single entity (instead of FICO scores or other syndicated, aggregated measures) is limited by the ability of employing these professionals, and they are in short supply. An even greater problem, which I've found after years of trying with my clients, is leaping the crevasse of trust between predictive modelers and executives, who are always reluctant to act on any conclusion drawn that deviates from their current practice or intuition. Predictive modeling is definitely not the killer app for BI, but the use of predictive models, especially embedded in operational analysis and Business Process Management, definitely is.

Neil Raden is the founder of Hired Brains, providers of consulting, research and analysis in Business Intelligence, Performance Management, real-time analytics and information/semantic integration. Don't miss Neil's many insightful articles in the Intelligent Enterprise archive.Predictive modeling isn't a crystal ball, and despite the efforts of Business Objects, Hyperion, Microstrategy and SAS to get predictive modeling into mainstream BI tools, there are many other reasons for its lack of success... Predictive modeling is different from using predictive models... Automating decisions with predictive models is a good idea, especially when those decisions are numerous and each one has a farily low risk of error.