All the research points in the same direction: data volumes are growing at a rampant rate within most enterprises, with estimates ranging from 20 percent per year on the conservative side to 50 percent per year among the largest organizations. The prognosis is such that "within a few years, traditional row-oriented relational databases are really going to be pushing their limits," predicts Boris Evelson of Forrester Research. “There are a lot of DBAs who don’t want to hear it because they’re so entrenched in Oracle and IBM DB2 and Microsoft SQL Server, but I think a new era is coming.”
Enter data warehouse appliances and column-store databases. While the technologies are not new, the last few years have seen the appliance market take off, and in the last few months, column-store database options have multiplied. Venture capitalists are placing multi-million-dollar bets on both categories, and they are safe bets, too, given exploding data volumes and the corporate desire to do fast-yet-in-depth analyses of all available data.
“One of the major reasons that Wal-Mart, Staples and Amazon.com have been so successful is that they analyze their data, and they have it at the fingertips of the entire enterprise,” says Foster Hinshaw, CEO of upstart data warehouse appliance vendor Dataupia and a cofounder of Netezza. “You have to be able to drill down on all your data and understand what it means in terms of where you site a new store, what SKUs are moving, what product you put in different locations and what customer programs you offer tomorrow.”
While the appliances have a head start in wooing business away from the leading data warehouse vendors, in the right applications, column-store databases can offer a higher-performance and more cost-effective alternative. What's more, column-store database vendors are beginning to offer appliance configurations that offer speed- and ease-of-deployment advantages. This article looks at the promise and progress in column-store databases, the limitations of the technology and when appliances built on conventional row-based databases make a better choice. We also offer five dos and don'ts you should apply in any appliance/column-store database buying process.