Gartner BI And Data Warehouse Quadrants: More Science, Less 'Magic' - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management
07:06 PM
Doug Henschen
Doug Henschen
Connect Directly

Gartner BI And Data Warehouse Quadrants: More Science, Less 'Magic'

Gartner's 2012 business intelligence and data warehousing vendor analyses reveal more customer survey work, fewer dark arts.

Back in seafaring days of yore, ship captains with unsavory crews made a habit of adding all sorts of phony steps to the process of using the sextant, compass, watch, and chart, lest potentially mutinous hands catch wise to the otherwise straightforward science of navigation.

They'd take multiple sextant readings of no consequence, swing their pocket watch like a pendulum, counting meaningless beats, then scratch down nonsensical calculations. The show made the task of choosing a safe course seem like a mysterious art that only the learned captain could handle.

Flash forward a few centuries and consider the Gartner Magic Quadrant, the analyst firm's popular charting of IT vendor leadership. How does Gartner go about triangulating the vendor dots on the horizontal "completeness of vision" and vertical "ability to execute" axes? Detractors have taken their shots at the process over the years, comparing the Magic Quadrant to a Magic 8 Ball or dartboard, even accusing Gartner of favoring "whoever pays the most."

Gartner last week published its business intelligence (BI) and data warehousing (DW) quadrants, and I have to break with the pundit tendency to bash these reports. In fact, I'm not the only observer who has noticed a bit less dark art and a bit more transparency in Gartner's process--at least where the BI and DW reports are concerned.

[ Want more on business intelligence and the data platform? Read our 2012 BI & Information Management Report. ]

Gartner uses a well established methodology (explained in this FAQ) across all quadrants. It entails a process of (typically annual) vendor briefings and surveys followed by customer surveys and interviews. The teams of analysts who write up each report stick to the general methodology, but they also develop market- and product-specific criteria, and they're free to put a greater emphasis on, say, the customer or vendor surveys, depending on the maturity of the market in question.

Gartner's BI Quadrant, which is available through this Tableau Software link, has relied much more heavily on customer surveys in recent years. Gartner surveyed 1,364 customers this year, and its findings were applied to three out of seven criteria behind the vertical "ability to execute" axis (namely, assessments of the product/service, sales execution/pricing, and customer experience).

Over the last three years, Gartner has also published the results of the customer survey, giving practitioners and vendors alike a chance to read verbatim quotes and scoring results. "We think the [Quadrant] process is definitely becoming more transparent over time, which is good for all the vendors and the customers who rely on this information," says Dan Jewett, VP of product management at Tableau Software, one of two "challengers" in this year's BI report (along with Tibco Spotfire).

The survey write-ups are particularly useful, says Jewett, because tech buyers can see how different vendors stack up on the attributes they're most concerned about.

Some vendors aren't so keen on Gartner's customer survey approach. SAP, for one, was somewhat dismissive of this year's BI quadrant as "a subjective ranking system with a heavy weighting on a small sample of customer survey responses," as described in a statement by Jason Rose, SAP's VP of business intelligence marketing.

What's surprising is that SAP is one of eight companies Gartner placed in the BI leaders' quadrant. But the company had the lowest "ability to execute" among those eight, ranking below Oracle, MicroStrategy, Microsoft, Information Builders, QlikTech, IBM, and SAS, in that order.

I, for one, don't consider 1,364 to be a small sample, especially considering that Gartner's sample two years ago included fewer than 900 respondents, including 91 SAP BusinessObjects customers. SAP's dot was in a similar spot back then.

Data Warehouse Vendor Rankings

Gartner's DW quadrant is far less customer-survey-intensive than the BI quadrant, acknowledges co-author/analyst Donald Feinberg, but he says he and his peers are using more survey data than ever before. "We've not only encouraged vendors to send us more references, but we're sending more surveys to Gartner clients," he explains.

There weren't any real shocks in this year's DW leaders quadrant. Teradata, Oracle, and IBM, respectively, still lead the rankings. EMC and Sybase swapped positions (roughly), while Microsoft somehow slipped into leader's quadrant, despite the fact that Gartner says it has yet to see a production deployment of Microsoft's SQL Server Parallel Data Warehouse. (This last case is one of those instances where you sense a little "magic" at work.)

What some report readers find most puzzling about Gartner's quadrants is the fact that vendors never reach the upper-right-hand corner. In fact, Teradata, Oracle, and IBM are further from that corner this year than they were last year. This evolution, says Feinberg, simply reflects the fact that markets are dynamic and that expectations keep rising. One year ago, big data was of growing importance, but customers now expect a comprehensive vendor vision and ability to execute on that score.

"A lot of people make the mistake of thinking of the edges of the quadrant as fixed, when actually it's the center that's fixed," observes Tim Negris, VP of marketing at cloud-based data warehousing vendor 1010data, the lone "challenger" in this year's Gartner DW Magic Quadrant, which is available through this 1010data link.

In other words, the average vendor vision on the horizontal axis and average customer expectations for ability to execute are dead center, but that's a moving target, explains Negris, who has also briefed Gartner analysts when he worked for Oracle and IBM.

None of this scrutiny would matter if nobody paid much attention to Gartner's quadrants, but it's not just vendors looking in the mirror, Negris says. "Judging by the standing-room-only crowds at Gartner conferences when they go through these reports in detail, customers are very interested because it helps them see what's happening," he says.

My point is that there's a lot of rigor behind the quadrants, and the BI and DW reports that I cover have only improved with the inclusion of deeper customer research. I know from first-hand experience that our own InformationWeek assessments, like our 2012 BI & Information Management Report, hugely benefit from in-depth end-user surveys.

As one critic of the recent Forrester Wave Report on Enterprise Hadoop Solutions advised, even if you take exception to the ranking of this or that vendor, you can get a lot of value just by looking past the chart and reading the content of the report.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
2/22/2012 | 12:42:13 PM
re: Gartner BI And Data Warehouse Quadrants: More Science, Less 'Magic'
Thanks for a look behind the scenes of the Magic Quadrant.

For an interactive MQ chart with the trajectories over the last 5 years (2008-2012) on BI see here:
2021 Outlook: Tackling Cloud Transformation Choices
Joao-Pierre S. Ruth, Senior Writer,  1/4/2021
Enterprise IT Leaders Face Two Paths to AI
Jessica Davis, Senior Editor, Enterprise Apps,  12/23/2020
10 IT Trends to Watch for in 2021
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/22/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll