The 489-page "The BI Survey 8" covers 23 products in total. The most prized survey findings -- those that prompt would-be software buyers to shell out $4,995 for the report -- are the eight aggregated ratings and 26 product-by-product measures of attributes including selections wins, deployment depth, success rates, implementation times, product reliability and customer loyalty, among other measures. Survey publisher Business Application Research Center (BARC) won't allow journalists to share any of the more than 50 dashboards detailing these results, but Pendse says customer sentiment in recent years has favored single-product independents while customers of the acquired mega-vendors have expressed frustration.
"Most people have been pretty disappointed by the results of those acquisitions of a year or two ago," Pendse says. "In most cases the next big releases [following those acquisitions] either came very late, didn't happen at all or didn't meet up with expectations." He cites a Hyperion upgrade touted before Oracle's acquisition for which "the main focus turned into integration with Oracle tools that the average Hyperion customer doesn't even have."
Similarly, the gap between customer perceptions and vendor perceptions of product-support quality was most acute among the mega-vendors. "The companies that are doing the acquiring or getting acquired delivered way worse product support than the one-product independent vendors," Pendse says. "Business Objects customers in particular complained about poor-quality support." Indeed, Business Objects (followed by Hyperion) had the lowest product-support-quality score among the 23 products and vendors ranked.
In contrast, independent products from Panorama, Information Builders, QlikTech and MicroStrategy had the highest overall scores across the 26 product quality, success, support and loyalty attributes measured.
The report notes many independent vendors cooperated with and supported the survey by encouraging customers to participate in the Web-based surveys. It also acknowledges that SAP, Oracle and SAS did not support the survey drive. Did this affect the outcome of the survey results?
"We didn't get as many SAS customers as we would have liked, but we did get good [statistically valid] samples of SAP, Hyperion and Oracle customers," Pendse says.
And given that the surveys were conducted from June to September 2008, has the value of the research been diminished by drastic changes in the global economy following the research period? "The Survey is about what has happened over a long period, not just the most recent purchases and implementations," Pendse responds. "If we are to speculate, I think it's a safe bet that new BI projects are likely to be smaller and more pragmatic, rather than grandiose -- which is a development I welcome. The Survey results confirm smaller, faster projects are much more successful than attempted strategic BI projects, which take too long, cost too much, deliver too-little and often fail."