Q&A: Ambuj Goyal on IBM's Path to Prediction and Optimization

Touting new industry- and domain-specific alternatives, Big Blue's Information Management GM says predictive and statistical modeling are being used as "a hammer" for many business problems for which they are not the right tool.
You're clearly challenging the likes of SAS and SPSS, which are known for tools and models that are applied in many different scenarios and industries, but these vendors also have industry- and domain-specific applications and long-standing relationships with companies in lots of vertical industries. Are you suggesting their technologies are being misused?

Predictive modeling has become a hammer for these vendors, and in many cases it's overkill. For example, we were recently talking to a client that has created a 100-terabyte model, using software from one of those vendors, to do prediction. They're using an amazing amount of computing capacity because the only method that they had available was a statistical model coming from one of these vendors. But the problem could have been solved much more simply with a policy engine, which could have been created for less than $1 million rather than requiring tens of millions of dollars of expense.

So I'd say these vendors have been using what they have as a hammer regardless of the business problem at hand. As we've taken this problem-specific and domain-specific approach, we find that different techniques are the right ones to solve a business problem. In many cases, people don't need to spend the kind of money they are currently spending to be able to solve their problem.

Are those vendors very good in the data mining capability of predictive models? Yes they are very good in the statistical domain of predictive models. Are they very good at constrained logic programming or Markov decision processes or rules engines? No, they are not.

In what areas would you give "the hammer" its due, and what would you say to all those professionals out there who are applying various analytic techniques using tools from the likes of SAS and SPSS?

You can apply those tools in any domain, and in many cases they do a very good job. But it can also be overkill... A combination of techniques can sometimes solve the problem better and less expensively.

Are there companies out there that understand all the techniques and combinations of techniques you're talking about, or is that insight that can only be obtained from IBM Global Services Consultants?

Many people and organizations are starting to embrace our approach. In one of the showcases that we ran at our Information On Demand conference, there were about nine global systems integrators highlighted who are leveraging our Information On Demand stack to solve specific problems in specific domains. And a huge number of our partners are also deploying these techniques.

Many view the real bottleneck in spreading the use of advanced analytic techniques as being the scarcity and expense of experienced analytics professionals. Did that factor in IBM's prepackaged, industry- and domain-specific approach?

You're right that there's not enough talent out there. But companies also don't always care [about having that level of expertise]... They just want to solve a business problem, and they may feel that they don't need to have deep technical experts who know how to use the hammer.

In each of the industries we have addressed, and really in every industry, there are only three things that people are trying to do to leverage information. One is getting better operational efficiency. Second, they are trying to do a better job at compliance. And third, they are trying to do a better job of taking care of customers and partners. In each of these three areas, we have identified top issues, so in automotive, for instance, there are three top issues: sales operations and planning, partner management and trade-up management. Those are the three areas where the industry is struggling, so we have captured those into a set of solutions with KPIs, dashboards and a set of underlying analytic techniques. People can deploy it without the huge amounts of expertise associated with creating a 100-terabyte model for doing prediction....

From our point of view, this approach is bridging the gap between business need and technology, and we can help companies take a simpler, less expensive approach.

Editor's Choice
Brandon Taylor, Digital Editorial Program Manager
Jessica Davis, Senior Editor
Terry White, Associate Chief Analyst, Omdia
Richard Pallardy, Freelance Writer
Cynthia Harvey, Freelance Journalist, InformationWeek
Pam Baker, Contributing Writer