Gartner Ranks Data Warehousing Leaders

Demand for data warehousing (DW) technologies held steady in 2010 even as many other IT categories retrenched in a tough economy. That's the macro take detailed in Gartner's just-released "Magic Quadrant for Data Warehouse Data Management Systems."

Despite steady demand, the DW market is far from static. Three key players, Greenplum, Netezza and Sybase, where acquired in 2010, and the consolidation is likely to continue, says Gartner. What's more, column-store, in-memory and cloud-based databases may have a disruptive impact on the market.

What has been stable has been the top three spots in Gartner's Magic Quadrant (MQ). Teradata is once again in the top-right position, highest on the vertical "ability-to-execute" axis and farthest right on the horizontal "completeness of vision" axis. It's followed by Oracle and IBM, which have been in the number-two and number-three spots for years.

The MQ details "strengths" and "cautions" on every vendor included in the report. Teradata's strengths include its flexible systems management software and its mature capability to integrate advanced analytics, says Gartner.

Teradata has positioned its 2650 appliance against competitors IBM Netezza and Oracle Exadata, but Gartner cautions that customers aren't quite clear on when to choose the appliance over Teradata's enterprise-class platform. What's more, the appliance doesn't always stand out in performance when tests and planned uses don't involve highly mixed workloads -- long a strong suit for Teradata.

Oracle strengths include its commanding (48%) share of the relational database market and its Oracle RAC (Real Application Clusters) technology. Gartner says RAC ensures high availability while also serving as the foundation for scaling out the Exadata Database Machine. Oracle eased administration with its 11g database upgrade through improved materialized view and cube management and an added Partition Advisor that suggests best configurations to maximize performance.

Gartner described the 11g upgrades as strengths, yet it also cautions that many customers say Oracle database administrative demands are higher than some competitive products. Another Oracle caution: Watch out for contract and pricing practices including "high prices, uneven and wide-ranging discounts, increasing software audits, high cost of maintenance and reluctance to negotiate on renewals," according to the report.

IBM's DB2-based products and acquired Netezza products are covered separately in the MQ report (with the former ranked third on the quadrant). IBM is the only database vendor that can offer an information architecture spanning all systems, Gartner notes. That includes OLTP, DW and retirement of data (the last covered by Optim products). IBM has also embraced Hadoop through InfoSphere BigInsights, which Gartner says gives DW managers confidence that IBM products are evolving to meet new demands.

The key caution on IBM's DB2 portfolio? There don't seem to be enough architects and DBAs to go around. This seems to be confirmed by the fact that IBM has been more selective about the projects it goes after. Not counting Netezza, IBM's market share declined about 0.7% while Oracle's declined about 1.8% in 2010, Gartner reports.

The three other vendors in the MQ top-right leaders quadrant are the three key vendors acquired in 2010: Sybase (by SAP), Netezza (by IBM) and Greenplum (by EMC). Sybase is a bit more of a visionary, as positioned by Gartner, while the last two are in fairly equal position. Challengers and Visionaries

With its release of the SQL Server Parallel Data Warehouse (PDW), Microsoft stands out in Gartner's top-left "challengers" quadrant (meaning it's high on the vetical ability-to-execute axis but less than halfway across the horizontal completeness-of-vision axis). Open-source vendor Ingress also nudged into the challenger realm, but mostly on the strength of its new (and entirely commercial) VectorWise analytic database.

Microsoft's PDW resolves some of the scaling issues with Microsoft SQL Server as a DW platform, but clustering "remains a more manual process than with some of its competitors." Indeed, Gartner cautions that customers report difficulty with clustered server performance and high availability in certain configurations.

The "visionaries" in the bottom-right quadrant of this year's DW report include, in order, Aster Data, Kognitio and ParAccel. Ranked far-right for their completeness of vision but low on ability to execute (given their small customer bases, employee numbers and service-and-support footprints), these players are prime targets for acquisition.

Indeed, Gartner sees continued consolidation in the DW market, and these visionaries (and a few other small vendors outside the visionaries quadrant) could "reduce development efforts and accelerate functionality" for an acquiring larger vendor.

Trends to Watch

On the technology front, Gartner's MQ and related "Data Warehousing Trends for the CIO" report, released this week, foresee broader use of column-store architectures, even as a main data warehouse platform. This is significant in that column-store has long been views as an analytic data mart technology, but Gartner says Sybase in particular has made strides to support enterprise data warehouse deployments.

8 Big Data Deployments In Detail

Gartner expects some column-store pricing models to get more aggressive, with pricing based on data volumes rather than per-user or per-core measures.

In-memory databases (some from BI vendors, some from DW players) are delivering extremely fast query response times. This will have a disruptive effect on data warehousing, Gartner predicts, with a "higher probability that analytics and transactional systems can share the same database."

This last bit is clearly a reference to SAP's strategy to support its applications and fast in-memory/column-store-powered analytics on the same database. InfoBright, Kognitio and Vertica also boast in-memory capabilities, and this boosts their prospects as acquisition targets.

Most DW players only address private-cloud deployments as "an emerging infrastructure design choice," says Gartner. Notable outsourced data warehouse and Data Warehousing as a Service (DaaS) options include 1010data and Kognitio, the latter with a partnership with CapGemini. Gartner see demand for cloud-based options increasing, but there's no side-stepping the always-present data model development and data integration challenges. Get A POC

Gartner's key advice to would-be DW buyers is to perform proof-of-concept (POC) assessments among a few finalist candidates. These tests should be done at the customers site with the customer's data and with as many data sources, users and simulated workloads as possible.

Most buyers heed this advice, says Gartner, but some vendors make things difficult. IBM, for example, has been pickier about participating in POCs while Oracle avoids on-site POCs entirely, pushing customers to perform such tests at one of nine international Exadata test sites. Given Oracle's market share, you could imagine on-site POC demands getting overwhelming.

What customers increasingly need is the ability to handle mixed workloads and to optimize performance for specific needs. Gartner encourages thorough assessment across six capabilities: bulk/batch loading, reporting, online analytical processing (OLAP), real-time/continuous load, data mining and operational BI.

Top 15 Data Visualization Tips

To optimize these workloads, hardware management for input/output, disk store and CPU/memory balancing are now included "as a matter of course" in DW platforms, Gartner notes, but it's important to explore capabilities and flexibility.

Storage optimization and compression are also receiving a lot more attention. EMC has stepped forward on storage management and Oracle and the column-store vendors on compression.

Vendors are putting a lot of performance and technology claims out there these days, but these differentiators "are not necessarily significant to the use case," Gartner cautions. That's why POC with your data, your workloads and your user expectations are vital to success.

A complete copy of Gartner's Magic Quadrant report is available from this link at the Sybase Web site.