Your Accuracy May Vary
The accuracy of predictive analytics hinges on the complexity of the situation being assessed and the number of other variables. In other words, a lot can go wrong in the process of presaging. "Let's be real," says Valen Technologies' Vlasimsky. "It will never be clairvoyant."
The day will never come when we can predict the outcome of the stock market, says Lutz Hamel, a professor of computer science and statistics at the University of Rhode Island. There are simply too many variables that change too quickly. On the other hand, Wall Street firms can predict short-term trading trends--that's what automated trading is all about--and many make a nice profit doing just that.
Tom Wicinski, managing director of customer marketing analytics at FedEx, will happily take the 65% to 90% accuracy rate he says the package-shipping company's predictive analysis system is providing. FedEx uses SAS Institute's Enterprise Miner and other tools to develop models that predict how customers will respond to price changes and new services, which customers are at risk of jumping to a competitor, and how much revenue will be generated by new storefront or drop-box locations. Accuracy, Wicinski says, depends not just on a problem's complexity and the number of variables, but also on the amount and quality of the supporting data.
FedEx began using predictive analytics for customer prospecting in the 1990s. But the company has broadened its use of the technology, applying it to more complex business problems. Applications, including the customer-at-risk system, are relatively new. "It's becoming a more mainstream business process," Wicinski says.
FedEx next will deploy predictive analytics in real-time operational settings such as call centers, he says, helping customer service reps identify at-risk customers and take the necessary steps to make them happy. Today, FedEx call-center agents and other front-line personnel must alert a sales rep when red flags go up--and that process may not be fast enough.
Financial results always are a good indicator of success. Alumni donations to the University of Utah's David Eccles School of Business increased 73% last year after the school used predictive analysis software from Kintera to determine which of the 300,000 people in its alumni database were most likely to respond to its annual appeal for donations. "It's always a question of who do we want to reach given the limited resources we have," says Erika Marken, development research director at the university.
While predictive analytics technology is most prevalent in financial and marketing applications, it's expanding into areas such as health care and crime prevention and even counterterrorism.
In Richmond, the predictive analysis system police began using two years ago determines where and when crimes are most likely to occur using a database of past calls to police, arrests, and crime incidents--some going back 15 years. The system also factors in weather data, and it tracks local festival, sporting, and other events. The system comprises SPSS's Clementine predictive analysis software, reporting and visualization tools from Information Builders, and predictive models developed by RTI International, a research organization.
Police commanders can query the system about specific crimes, such as determining which neighborhoods are most likely to experience armed robberies or auto thefts. For example, the police have zeroed in on armed robberies in nightclub parking lots near closing time--robbers consider inebriated club-goers to be easy marks, says Colleen McCue, a senior research scientist at RTI.
As more data is added to the system, accuracy should improve, Chief Monroe says. But it has its limitations. The analysis is primarily restricted to time, place, and type of crime, while details such as the type of weapon used in past crimes aren't considered. And the predictive models must be updated with new information, such as increases or decreases in the types of drugs being sold on the streets.
When it comes to homeland security, details about how government agencies are using data mining and predictive analytics aren't easy to get. But work under way at the Pacific Northwest National Laboratory provides some hints. As part of the coun- terterrorism work done under the departments of Defense and Homeland Security, the lab is combin-ing predictive analytics with visualization technol-ogy for trend analysis and pattern recognition to detect signs of an impending terrorist attack. Sen- ior program manager Steve Martin is circumspect about how exactly such applications would be used, but he says it's likely the feds are using the technology to analyze phone call patterns.
The lab also is combining predictive analytics and behavioral analysis in the belief that terrorists might be caught on security cameras and identified through their behavior--if they loiter in a specific place, for example--before they have a chance to carry out an attack.