Column-Store Databases and DW Appliances: How to Make the Right Choice
With data volumes exploding, conventional enterprise data warehouses are fast running out of headroom. Data warehouse appliances are starting to fill the gap, but the emerging category of column-oriented databases may offer a better option. The key to success is matching your application to the right product.
All the research points in the same direction: data volumes are growing at a rampant rate within most
enterprises, with estimates ranging from 20 percent per year on the conservative side to 50 percent per year among the largest organizations. The prognosis is such that "within a few years, traditional row-oriented relational databases are really going to be pushing their limits," predicts Boris Evelson of Forrester Research. “There are a lot of DBAs who don’t want to hear it because they’re so entrenched in Oracle and IBM DB2 and Microsoft SQL Server, but I think a new era is coming.”
Enter data warehouse appliances and column-store databases. While the technologies are not new, the last few years have seen the appliance market take off, and in the last few months, column-store database options have multiplied. Venture capitalists are placing multi-million-dollar bets on both categories, and they are safe bets, too, given exploding data volumes and the corporate desire to do fast-yet-in-depth analyses of all available data.
“One of the major reasons that Wal-Mart, Staples and Amazon.com have been so successful is that they analyze their data, and they have it at the fingertips of the entire enterprise,” says Foster Hinshaw, CEO of upstart data warehouse appliance vendor Dataupia and a cofounder of Netezza. “You have to be able to drill down on all your data and understand what it means in terms of where you site a new store, what SKUs are moving, what product you put in different locations and what customer programs you offer tomorrow.”
While the appliances have a head start in wooing business away from the leading data warehouse vendors, in the right applications, column-store databases can offer a higher-performance and more cost-effective alternative. What's more, column-store database vendors are beginning to offer appliance configurations that offer speed- and ease-of-deployment advantages. This article looks at the promise and progress in column-store databases, the limitations of the technology and when appliances built on conventional row-based databases make a better choice. We also offer five dos and don'ts you should apply in any appliance/column-store database buying process.
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.