Candid Thoughts About Recent Gartner & Forrester Research

My reaction to recent Gartner and Forrester analyst reports, as covered in Intelligent Enterprise articles, is that the firms are tarting up a couple of instances of questionable research.

A Gartner report urges us to "Reduce Costs of Data Integration by Rationalizing Tools and Infrastructure, and Centralizing Skills." According to Gartner's press release, "organizations can save up to $500,000 annually by rationalising tools in the short term and adopting a shared-services model in the longer term," a simplistic statement to say the least, more on which in a moment.

Forrester's report, "Voice of the Customer: The Next Generation," does not seem simplistic, but I do get to wondering when Voice of the Customer research is based on interviews with "more than 20 companies" and at least 17 of the ones listed are VOC solution providers. I asked lead author Bruce Temkin, "How did you confirm their assertions with user organizations?" He did not respond.The report is not freely available from Forrester -- Temkin declined to provide me a copy himself -- so I'll refer you to Antone Gonsalves' Intelligent Enterprise article, "Six Trends Changing Customer-Feedback Programs," for coverage.

To help you in your assessment of Forrester's approach to research, consider Jeff Kelly's SearchDataManagement.com article, "Business intelligence software adoption lags BI vendors' perception." Kelly reports on a survey conducted by estimable BI analyst Nigel Pendse that found, as reported by Kelly, "8% of workers at organizations that have deployed BI tools make use of them, on average, but vendors put that figure among their customers at 14%. Likewise, 27% of users rate their vendors' BI support services as 'excellent,' while the vendors themselves believe they are doing an excellent job in 51% of cases." Kelly quotes Pendse, "'I think the vendors are deluding themselves. They genuinely think their products are better than they are.' He added that many vendors don't actually interact with their customers enough to truly understand their deployment and use issues."

Myself, I believe that Pendse's analysis applies broadly to all IT domains, which would include the VOC arena covered by the recent Forrester VOC report. If I am correct, what should we infer about these research findings based, apparently, almost exclusively on vendor interviews?

And now back to the Gartner data integration report.

The Gartner report is also not freely available but Antone Gonsalves covers it in his story, "Poor Use Of Data Integration Tools Can Waste $500,000 Annually: Gartner." See also the Gartner press release announcing the report. The opening sentence of that release states, "Organisations that have implemented substantial data integration architectures can save more than $500,000 annually by rationalising tools in the short term and adopting a shared-services model in the longer term." Wow! What a laughably simplistic assertion. "Organisations... can save more than $500,000 annually." So that applies to Unilever with €40 billion in 2008 sales and it also applies to a Mom & Pop operation with turnover of a few million?

Gartner does preface its overreaching, blanket claim with a redeeming, restrictive condition, that findings apply to organizations "that have implemented substantial data integration architectures." That's a sensible restriction: an architecture enables a rationalization process. But tell me, what proportion of organizations have implemented a "substantial data integration architecture"? I can't say and you probably can't and the rest of that Gartner release doesn't say. It in fact goes on to ignore that sensible condition that it asserted up top. Oh, and how much should an organization (large or small) expect to spend creating a "substantial data integration architecture" in order to be in a position to realize $500,000+ in savings?

Rationalization should be about making sure you're getting all the data you need for operations and analyses from the originating systems to the target systems in a timely, complete, clean, and functional manner. That is, start by understanding the task and the tools you're using to make it happen. Create an architecture (but do be pragmatic about it rather than doctrinaire, as I'd infer the Gartner folks would have you be).

Contrast my view with that of Ted Friedman, vice president and distinguished analyst at Gartner: "The first step is for IT teams focused on data integration to save money by rationalising tools." Yup. You shouldn't look first at the job at hand. You shouldn't start by talking to the business folks. If your focus is data integration, says Gartner VP and distinguished analyst Ted Friedman, your priority should be to save money. And lucky us, Gartner gives us a target: $500,000. If you haven't saved $500,000, then (implicitly) you haven't done the job.

I'll close by thanking technology writer Stephen Swoyer for tipping me off to the Gartner research. Steve sent me, Mark Madsen, and a vendor contact a note asking our reactions. (I'm solely responsible for mine.) I'm hoping these fellows will help me get out a survey on "Advantages and Disadvantages of Rationalizing Industry Advisory Services." The question at hand: "How much can organizations save by reducing their reliance on multiple, redundant industry advisory services in favor of open-source intelligence sources including independent research and analyses?" I'll poll all the independent IT analysts. Does $500,000 annually sound believable? I think I'll go for it.My reaction to recent Gartner and Forrester analyst reports, as covered in Intelligent Enterprise articles, is that the firms are tarting up a couple of instances of questionable research.