Is Gartner's Quadrant the Problem, Or Is It How It's Used?

Bashing Gartner's Magic Quadrants seems to be a popular industry pastime, but in truth, I kind of like the quadrants. My biggest gripe is in how the quadrants are used, not necessarily the quadrants themselves.

There are several positionings I could find fault with in the latest BI platforms Magic Quadrant (MQ). That both Microsoft's and Oracle's "ability to execute" is higher than IBM Cognos' and SAP BusinessObjects' would be two; I say that in terms of their BI-specific offerings, not their overall ability to execute. Vendors are very much at the mercy of the Magic Quadrants, when in reality, positioning in the MQ says little about how well a particular company or product will meet a customer's needs. And therein lies the problem.Far too many customers use the Magic Quadrant for creating an initial vendor short list. Customers will only evaluate vendors in the top right quadrant. For a vendor to appear anywhere else is really the kiss of death, particularly if a vendor is in the bottom left.

And yet, some vendors appear in other corners of the quadrant purely because they create a particular kind of BI tool. Tibco Spotfire and Tableau, for example, are not really BI platforms, so it's misleading to compare them to some of the broader solutions. This doesn't mean that customers should ignore these vendors. Instead, they should think about the types of applications and user requirements that are better served by these "niche" vendors.

Some vendors will tout that they "are rated on Gartner's MQ,", as if merely getting on someone's radar is cause for celebration. But again, it's how vendors and customers perceive this ranking. Customers assume if a vendor isn't on the MQ, then they must not be worth looking at. However a vendor's absence could mean many things: the vendor is too small; the vendor did not spend enough resources educating Gartner about their product and vision; or their focus isn't broad enough to be a good fit for inclusion in a particular MQ. Vendors who are favorably positioned on the MQ will only further add to the confusion suggesting customers should buy from them, because if Gartner rates them so highly, they are a safe and sound choice.

Customers like and need short cuts in figuring out when to look at particular products and vendors. The MQ provides that. It also seems to me that Gartner has gradually tried to improve the objectivity and transparency by which they position the vendors. However, cstomers need to know the limitations of the MQs and recognize that some poorly positioned or not-rated vendors may deserve consideration. Those upper right vendors may fulfill a number of strategic requirements, but it still reveals nothing about how well they will fulfill your precise requirements or how well they will fit within your technical architecture. That is why we continue to enhance the BI Scorecard and track those details.

Customers should use the MQ as just one of many data points in their evaluation process. Vendors could do well to help educate customers about how to use the MQ appropriately, rather than just touting it in the years they like their positioning.

Regards, Cindi Howson, BI ScorecardBashing Gartner's Magic Quadrants seems to be a popular industry pastime, but in truth, I kind of like the quadrants. My biggest gripe is in how the quadrants are used, not necessarily the quadrants themselves...