Predictive Analytics 101: The Limits of Intuition

One of the more engaging presentations at this week's TDWI Executive Summit was a helpful session entitled "The Yin and Yang of Implementing Predictive Analytics," presented by Matt Schwartz of Corporate Express and John O'Carroll of Capital One Auto Finance. Schwartz was the Yin, presenting on the office supply company's 18-month entry-level foray into prediction. O'Carroll was the Yang, a seasoned developer of highly complex models around customer segmentation, marketing campaigns and lending risk. Despite his focus on beginners, Schwartz engaged everyone, particularly the retailers and online marketers present, with a tutorial based on a Corporate Express market basket application."When people order a stapler on CorporateExpress.com, what else is going to be in their shopping cart?" Schwartz asked the audience. Many of the responses seemed obvious: staples, staple removers, tape, paper, paper clips, scissors and so on. Schwartz made the point that when doing predictive modeling for cross-sell/up-sell, it's best to temper pure intuition and rely on tests. "Opinion and gut feel can take you down the wrong path," he said.

Of the many related items measured in a probability-based analyses of lift, support and confidence, "we found that the number-one lifted item among customers buying staplers was a mesh pencil cup," said Schwartz." The mesh pencil cup was ordered 9,000 times. The stapler was ordered 10,000 times, and they were ordered 283 times together. It turns out customers were 12 times more likely to purchase the pencil cup when the stapler was already in the basket. The intuitive interpretation of this result is that Corporate Express customers often buy staplers for new hires, and pencil cups are a natural fit in a new-employee bundle. The company has since revised the "related items" displayed across many products to include obvious as well as not-so-obvious selections.

"We've seen a lift in average order size as well as improved stickyness on the Web site," says Schwartz.

Corporate Express late last year won a TDWI Best Practice Award for 2007 for this market basket application.

O'Carroll detailed some of the approaches and organizational best practices employed an an organization that sends out 100 million auto loan solicitations per year. The goal is to "make sure we extend the right offer to the right people," he says.

It seemed a very different and daunting domain (compared to the experiments Schwartz described). "I've seen models take as long as 18 months to develop, test and deploy," said O'Carroll. That's the world that has scared off many would-be practitioners of predictive analytics, so it was refreshing to hear encouragement and productive examples of how you can take baby steps into a powerful domain.In a TDWI Executive Summit presentation entitled "The Yin and Yang of Implementing Predictive Analytics," Matt Schwartz of Corporate Express presented the "Yin," exploring the office supply company's 18-month foray into prediction. He asked attendees, "when people order a stapler on CorporateExpress.com, what else is going to be in their shopping cart?" The answer might surprise you.