When vendors make it into the top-right "leaders" quadrant of a Gartner Magic Quadrant (MQ) report, they generally don't complain. But SAP isn't thrilled with the "ability to execute" positioning of SAP BusinessObjects in the 2010 Gartner MQ for Business Intelligence Platforms, which was released early this week.
"If you look at the results, it's nonintuitive and nonsensical that we would have less ability to execute, than, for example, Microstrategy," said Franz Aman, SAP's vice president, intelligence platform product marketing.
Never mind that SAP BusinessObjects ranked second on the "completeness of vision" axis (after IBM Cognos). SAP was so upset with the ability to execute ranking (where it was behind Oracle, Microsoft, IBM, SAS, and Microstrategy, in that order), that it went to the extreme of commissioning a blind survey of 24 CIOs, conducted by the Gerson Lehrman Group, that shows a higher ranking for SAP Business Objects based on an open-ended, respondent-defined interpretation of "ability to execute."
"We basically ranked in the ballpark of Oracle and IBM in the blind study, and the other vendors were in far distant spots," Aman said.
Gartner says it defines and measures ability to execute across all MQ reports in terms of seven dimensions: product and service competitiveness, vendor viability, sales execution and pricing, market responsiveness, market execution, customer experience and operations. The weighting on each of these seven dimensions varies from year to year based on customer inquiries and market changes, said Rita Sallam, the primary author of this year's BI MQ. She defended the ability to execute rankings saying they were based in large part on nearly 900 customer interviews, including 91 SAP BusinessObjects customers, most of whom were referred by SAP itself. As stated in the MQ report, both IBM Cognos and SAP BusinessObjects lagged in the survey's customer experience ratings, which Gartner interpreted as a result of post-acquisition challenges.
"When there's an acquisition, there's often a lot of turmoil," Sallam said. "SAP, in particular, had significant challenges when it changed over [BusinessObjects] support [to SAP]... IBM cut over Cognos support to IBM in 2009, and it also realigned sales territories and changed the way Cognos customers buy products. Change by itself can have a negative impact on customer perceptions."
Oracle saw a similar decline it survey-based customer experience ratings in 2008 after its Hyperion acquisition, Sallam said, "but it now seems like they beyond that transition and they had significantly better scores for 2009 than in 2008."