Gartner Magic Quadrant Spots Data Warehouse Leaders

Who doesn't dive right to the vendor rankings when opening a Gartner Magic Quadrant report? So let's end the suspense right up front on the just-released 2013 Magic Quadrant for Data Warehouse Database Management Systems: Teradata once again is in the top-right position, but Oracle, SAP and Microsoft all gained significant ground in the "leaders" quadrant. The two other leaders are IBM, which improved its standing, mainly on the horizontal "completeness of vision" axis, and EMC, which lost some ground on the vertical "ability to execute" axis.

There's more said about leaders and "visionaries" later in this story. But first, what about Garter's big-picture trend analysis -- that part that often gets overlooked? For starters, Gartner projected 10% growth in the database management system market overall in 2012 and it noted "a significant increase" in the ranks of organizations seeking to deploy data warehouses for the first time. It also illuminated advanced "logical data warehouse" developments that are helping advanced practitioners to take advantage of big data. (Gartner's Magic Quadrant is available as a free download from Teradata, but the report was not directly sponsored by Teradata or any other participating vendor).

The increase in new data warehouse deployments is a healthy sign that more companies are getting serious about making data-driven decisions, but the last thing these newbies should do, Gartner advises (despite the fame of its MQ reports), is dive right into vendor analyses.

"Most first-time buyers attempt to select the data warehouse platform vendor first without thinking though architectural considerations or even looking at the data source types," Gartner analyst Mark Beyer, co-author of the report, told InformationWeek.

[ Who are the leaders in big data platforms and analysis? Read Big Data Revolution Will Be Led By Revolutionaries. ]

This is the number-one rookie mistake in data warehousing, according to Beyer. And once the platform arrives, newbies compound the problem by building out the warehouse based only on known queries. "If you do that, you usually end up redesigning the whole warehouse within five years," Beyer said, noting that new queries and query types inevitably emerge along with new products or new lines of business.

The second-biggest (and closely related) mistake is building out warehouses based only on currently available data. When new sources, data types, required level of detail or part of the business emerges, the data model breaks.

"The key is understanding and modeling business processes before you settle on architecture and build out the warehouse," Beyer said. This exercise exposes where business activities converge and where data redundancies, conflicts and quality problems often crop up, Beyer said. If you look only at logistics and distribution, for example, but you don't consider warehouse management or inventory-management sources and activities, you're in for trouble.

"You don't have to model for every type of data right away, but you do have to understand where there are handoffs within the company," Beyer said.

A third rookie mistake is planning data-integration architecture in a linear, source-to-target path. Here, too, change is inevitable, so think in terms of modular tasks or tiers so you can flexibly mix, match, rearrange and insert new steps into an overall process. Individual tasks might be detecting data changes in a source, parsing the data or performing quality checks.

Advanced data warehousing practitioners may have their routine data-modeling and data-integration ducks in a row, but like many, they're struggling to make use of the incredible volume and variety of information associated with big data. This is where Gartner's logical data warehouse vision comes into play. The vision has emerged because the old idea of creating one, all-encompassing enterprise data warehouse has seldom worked out. What's more, users have been frustrated by delays in bringing new data sources and related, lower-latency analyses into production.