Data Warehouse Appliances: Are They Simple or Complex?

In these recessionary times, giant convention hotels can feel like empty cathedrals. At the recent TDWI World Conference (August 2-7), the Manchester Grand Hyatt San Diego looked as though it could have benefited from a little religious exhortation to draw people inside its halls. Of course, with the city and its surrounding waters basking in the bright summer sunshine, some might have entered, checked in and slipped out the back way to enjoy an ice cream in Seaport Village rather than settle in for some data warehousing instruction. However, they would have missed not only TDWI's expanding educational offerings -- from the "soup" of technical data warehouse design to the "nuts" of how executives can use information for business leadership -- but also evidence of significant shifts going on in the BI and data warehousing market.

From its appearance, the TDWI exhibition show floor looked pretty much as it always has. Familiar faces were in place at many of the booths, although not necessarily in the same booths as before. There was the usual contest to get your card stamped by the exhibitors; this time, if you got them all, you were entitled to a trip inside a little booth full of money being blown around by air hoses. You could keep what you caught: Not a bad deal in these tight times. If you're ever in one of those, from my observation the most effective ways to trap bills were to press them into your underarms, use your forearms to squeeze them against your chest and pinch bills against your collarbone with your chin. Yes, you look ridiculous, but you must be ready to hold that pose until the assistant plucks the bills off of you. Teeth were ineffective. I would not recommend trapping bills between the knees: someone fell over trying to do that.Walking down the aisles, you could sense change coming to the BI/data warehouse realm. Vendors have seen the future, and it is appliances. IBM had a prime spot to the left of the entrance. Not long before the TDWI event, the company introduced the IBM Smart Analytics System, a graduated series of data warehouse appliances that offer more tightly (or using IBM's term, "deeply") preconfigured software, storage, servers and systems management packages than the previous Balanced Configuration Units. While IBM has been touting the advanced technology that it will put into these bundles, Netezza's TwinFin announcement may have stolen the thunder. The marquee news, as Curt Monash discusses in his blog post, is the price: "less than $20,000 per terabyte," according to the company.

Supporting that revolutionary price are some major changes in technology implementation, including the use of "industry standard" (in this case, IBM) blade servers as part of Netezza's Asymmetric Massively Parallel Processing Architecture (AMPP). An important upshot of Netezza's use of commodity technology in the AMPP is that the systems will be more open and straightforward for application and data service development. Reducing the potential for isolation will help Netezza to move from "a single product company with a good idea to something bigger," said Phil Francisco, VP of Product Management and Marketing.

Into these appliances, IBM, Oracle (Exadata), Netezza and other vendors are pushing some of their most advanced database technology as well as extensive implementation of relatively new things for data warehouses. These include blades and Infiniband, the de facto industry standard interconnect that is supposed to take the place of proprietary technology for knitting together MPP architectures. Exciting as it is to see these technologies at work, there's still a learning curve going on regarding their performance and administration -- or at least in the implementation of "smart" technology designed to manage it in the appliance for you.

Thus, the irony about appliances is that even as vendors market them to be simple, plug-and-play systems, it is becoming increasingly important for customers, as they expand their implementations, to understand the products' innards and how they compare. What are their true capabilities and limitations? Can they handle multiple kinds of workloads? How exactly do they scale? Where are the hot spots? Do they have good administration tools that typical customers of appliances can use properly?

"We're counting on 'the influencers' to explain the technology advantages to customers," one vendor told me. These "influencers" -- consultants and other experts -- may well be measured by the depth of their appliance knowledge and how the different capabilities match up with the kind of BI or analytics companies have in mind.

Thus, data warehouse technologists probably need not hang up their star schemas and performance design tricks quite yet. Appliances are not yet Toyota Corollas; they may be more like BMWs. And so like sports car mechanics, professionals have to get under the hood -- at least until advanced, automated and probably as-a-service appliance management comes along. With business insight on the line, companies can't afford a breakdown.Vendors have seen the future, and it is appliances... The irony about appliances is that even as vendors market them to be simple, plug-and-play systems, it is becoming increasingly important for customers, as they expand their implementations, to understand the products' innards and how they compare.