13 Big Data Vendors To Watch In 2013

Who doesn't dive right to the vendor rankings when opening a Gartner Magic Quadrant report? So let's end the suspense right up front on the just-released 2013 Magic Quadrant for Data Warehouse Database Management Systems: Teradata once again is in the top-right position, but Oracle, SAP and Microsoft all gained significant ground in the "leaders" quadrant. The two other leaders are IBM, which improved its standing, mainly on the horizontal "completeness of vision" axis, and EMC, which lost some ground on the vertical "ability to execute" axis.

There's more said about leaders and "visionaries" later in this story. But first, what about Garter's big-picture trend analysis -- that part that often gets overlooked? For starters, Gartner projected 10% growth in the database management system market overall in 2012 and it noted "a significant increase" in the ranks of organizations seeking to deploy data warehouses for the first time. It also illuminated advanced "logical data warehouse" developments that are helping advanced practitioners to take advantage of big data. (Gartner's Magic Quadrant is available as a free download from Teradata, but the report was not directly sponsored by Teradata or any other participating vendor).

The increase in new data warehouse deployments is a healthy sign that more companies are getting serious about making data-driven decisions, but the last thing these newbies should do, Gartner advises (despite the fame of its MQ reports), is dive right into vendor analyses.

"Most first-time buyers attempt to select the data warehouse platform vendor first without thinking though architectural considerations or even looking at the data source types," Gartner analyst Mark Beyer, co-author of the report, told InformationWeek.

[ Who are the leaders in big data platforms and analysis? Read Big Data Revolution Will Be Led By Revolutionaries. ]

This is the number-one rookie mistake in data warehousing, according to Beyer. And once the platform arrives, newbies compound the problem by building out the warehouse based only on known queries. "If you do that, you usually end up redesigning the whole warehouse within five years," Beyer said, noting that new queries and query types inevitably emerge along with new products or new lines of business.

The second-biggest (and closely related) mistake is building out warehouses based only on currently available data. When new sources, data types, required level of detail or part of the business emerges, the data model breaks.

"The key is understanding and modeling business processes before you settle on architecture and build out the warehouse," Beyer said. This exercise exposes where business activities converge and where data redundancies, conflicts and quality problems often crop up, Beyer said. If you look only at logistics and distribution, for example, but you don't consider warehouse management or inventory-management sources and activities, you're in for trouble.

"You don't have to model for every type of data right away, but you do have to understand where there are handoffs within the company," Beyer said.

A third rookie mistake is planning data-integration architecture in a linear, source-to-target path. Here, too, change is inevitable, so think in terms of modular tasks or tiers so you can flexibly mix, match, rearrange and insert new steps into an overall process. Individual tasks might be detecting data changes in a source, parsing the data or performing quality checks.

Advanced data warehousing practitioners may have their routine data-modeling and data-integration ducks in a row, but like many, they're struggling to make use of the incredible volume and variety of information associated with big data. This is where Gartner's logical data warehouse vision comes into play. The vision has emerged because the old idea of creating one, all-encompassing enterprise data warehouse has seldom worked out. What's more, users have been frustrated by delays in bringing new data sources and related, lower-latency analyses into production. Big Data Evolution

The logical data warehouse vision encompasses data repositories, data federation/virtualization technologies and distributed data processes that are beginning to make it possible to access and combine data in different ways without having to work through IT departments or developers. Independent federation leaders include Composite Software and Palantir, according to Beyer. In addition, database management system vendors Teradata, IBM, Oracle and others have introduced external table capabilities -- a form of federation -- that let them access and incorporate data that is outside their databases into analyses. Among these external sources are new platforms including Hadoop clusters and NoSQL databases.

Gartner's latest research, including papers that have not yet been published, shows that 6% to 10% of cutting-edge practitioners (1.5% to 6% of the total market) have implemented combinations of big data, traditional data warehouses and federation technologies.

"The key is putting two or more of these sources through a single entry point, and the leading vendors in the Quadrant and leading [practitioners] have done it," Beyer said.

[ Who are the leaders in big data platforms and analysis? Read Big Data Revolution Will Be Led By Revolutionaries. ]

The payoff is bigger, broader access to data that can support deeper and lower-latency decision-making. The Magic Quadrant report envisions a future in which "big data becomes 'normal' and the logical data warehouse becomes the best practice."

Leaders and Visionaries

Teradata has the highest completeness of vision ranking in Gartner's Magic Quadrant, due to its support for the logical data warehouse vision with its Unified Data Architecture, which combines Teradata, AsterData and Hadoop. Teradata acquired AsterData in 2011 as an analytic platform that supports MapReduce, graph analysis and other forms of analysis against unstructured and multi-structured data, and in 2012 it added deeper support for Hadoop integration.

Oracle's vision is less complete, according to Gartner, but it has the highest "ability to execute" ranking in the latest report, testament to its sales successes in 2012. Many of the companies that are new to data warehousing are choosing Oracle because "they incorrectly perceive that Oracle has a straightforward appliance offering and that Teradata's and IBM's are more sophisticated and, therefore, tougher to understand," Beyer said. Despite that perception, "Exadata is just as sophisticated as the others ... but marketing is different than reality," Beyer said.

In another case of perception versus reality, Beyer noted that SAP's steep rise in both its ability to execute and completeness of vision in the Magic Quadrant is mostly due to Sybase IQ rather than SAP Hana. "Sybase IQ has been tightly tied into the SAP supply chain, and that has improved execution," he said. "And with IQ 15.4, they introduced the capability to create virtual work units in the processor plan separate from the storage plan." That new feature gives IQ greater flexibility to manage workloads and service levels tied to hot and cold data.

Among other vendors showing significant gains in Gartner's latest data warehousing assessment, ParAccel and Kognitio both jumped into the "Visionaries" quadrant. ParAccel recently scored a deal with Amazon to use ParAccel as the underlying database for the Amazon Redshift service. That deal has brought ample funding and the promise of long-term stability. Gartner cited Kognitio strengths including pricing flexibility, performance and logical data warehouse support through new integrations with Hadoop and partnerships with Hortonworks and MetaScale .

It's not yet ranked as a Visionary or Challenger, but Actian rose significantly in Gartner's ranking following a reorganization in 2011. In recent months Actian has pursued acquisitions that will surely bolster its "completeness of vision" in next year's Magic Quadrant.