Reinvent The Data Warehouse With Column-Store Databases And Appliances
These approaches beat out conventional databases in price and performance. Here's how to find products that fit with your company's data analysis needs.
Data warehouses are growing fast. Nearly 40% of organizations say data volumes are increasing as much as 50% per year, while 18% say their warehouses are doubling in size annually, according to an IDC survey. To cope with the growth, the underlying databases have to be carefully tuned, but even then, many midsize and large companies now face six-, seven-, or even eight-figure upgrades of legacy data warehouses.
Enter data warehouse appliances and column-store databases. Both have taken off in recent years, with venture capitalists placing bets on a slew of startups. They're safe bets, too, given the corporate desire to do more in-depth analysis of all available data.
"One of the reasons Wal-Mart, Staples, and Amazon.com have been so successful is that they analyze their data, and they have it at the fingertips of the entire enterprise," says Foster Hinshaw, CEO of upstart appliance vendor Dataupia and a co-founder of Netezza, the 8-year-old leader in the data warehouse appliance market. Companies must be able to drill into all their data to understand where to locate a new store, which products are selling, which need to be moved to different locations, and what programs to offer customers, he says.
Boasting fast query performance, ease of deployment, and prices as low as $10,000 per terabyte, appliances have been wooing business away from the leading data warehouse vendors. It's no surprise that IBM and Teradata have responded with appliances of their own, or that Oracle has optimized reference configurations for third-party hardware. But the incumbents have yet to respond to the threat from column-store databases, which can deliver the industry's fastest query performance on complex analytic queries.
It sounds like a promising new era for data warehousing, but buyer beware. Appliances and column-store databases aren't always suitable replacements for a conventional enterprise data warehouse, or EDW. In fact, these alternatives are most often used for data marts that off-load data-intensive applications from the EDW, thereby avoiding (or at least delaying) the need to replace the main data warehouse. Whether you're looking for an analytic data mart or your next data warehouse, here's what to look for beyond the dazzling price, scalability, and performance claims.
Column Store Database
Data Warehouse Appliance
CNX Data Warehouse Platform
Dataupia Satori Server
Greenplum Database G3
InfoSphere Balanced Warehouse E-Class
Netezza Performance Server
* Column-store databases also offered in appliance configurations on third-party hardware
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.