Predictive Analytics: What Have You Done For Me Lately?

In our economic malaise, it would be hard to blame anyone for desiring to be magically transported to better times. Could predictive analytics provide the digital fairy dust that brings them closer?

Changing the economy is a tall order, but predictive analytics could enable companies to hang tough during lean times by helping them identify the most important customers, focus resources on critical business processes and uncover fraudulent behavior before it causes serious damage. Last month's Predictive Analytics World conference in San Francisco brought experts together to discuss the business potential of predictive analytics inside larger trends in marketing and process optimization.

What does "predictive analytics" mean, anyway? It's probably best not to get too distracted by a marketing terminology dispute, but many of the conference presenters were asked to weigh in. Some saw it as a kinder, gentler expression for data mining, or "torturing the data until it confesses," as keynote speaker John Elder put it. Others, such as Usama Fayyad of Open Insights (and formerly Yahoo!'s chief data officer and executive VP of research and strategic data solutions) saw it as "an activity that you do when you do data mining." Just as the "data mining" term came about in part as a way of distancing this activity from the theoretical world (and unfulfilled dreams) of artificial intelligence, predictive analytics is even more about doing something productive, not just interesting.

According to Wikipedia, predictive analytics, "encompass a variety of techniques from statistics and data mining that analyze current and historical data to make predictions about future events." Elder adds an emphasis on producing a result within a tight time window to enable an organization to "do something that will make a difference right away." The lesson of the last six months, when so many business forecasts and models have become irrelevant, is that a "predictive" insight is most valuable in the here and now, not as a projection of what will happen in the hazy future.

Of course, timeliness is especially important for online businesses that are trying to guide customers toward transactions based on their behavior -- and soon, that of their friends and the communities to which they belong. "Why look just at search data," Fayyad asked, "or just your data and not also that of your buddies and how they behave?"

It was clear at the conference that a variety of businesses are frothing at the predictive potential of social networking data; they are slowed, however, by concern that tramping on etiquette and privacy matters could kill the golden goose. The objective is to gain a fuller understanding of how buying decisions are made and how one event influences other events.

The exhibit area at Predictive Analytics World featured SAS and SPSS, which is no surprise since they are the dominant data mining software providers. However, there was quite a bit of buzz about the open-source R programming language, which is being adopted by a variety of vendors and could present a challenge to the two mainstays (I should note that both SAS and SPSS are now making moves to incorporate R). Unexpected coverage of R earlier this year in the mainstream media by The New York Times added to the excitement. The conference hosted a well-attended meeting of the Bay Area useR Group (of San Francisco). TIBCO's Spotfire Miner 8.1 release implements S+, a derivative of the S language upon which R is also based. The release employs technology from TIBCO's September 2008 acquisition of Insightful. When integrated with Spotfire's BI and visualization tools, the technology is able to provide users with a usefully actionable context for predictive analytics. SAP (Business Objects) had a booth highlighting the BusinessObjects Predictive Workbench, but representatives made no secret that they are looking to bulk up in predictive analytics tools.

Anne Milley, SAS senior director of Technology Product Marketing, noted in her talk that W. Edwards Deming said that "the objective of taking data is to provide a basis for action." Predictive analytics is best understood in that context: how organizations can apply data insights to guide smarter decisions, behavior and processes in the here and now. If successful, the future will take care of itself.