Data Pressure Cooker

Though introduced with less fanfare, recent database innovations have been critical

The database management systems market no longer captures the buzz it did a decade ago, when the client/server revolution was in full swing and a competitive band of vendors enjoyed huge and rapid growth. The technology debates between executives at Oracle, Sybase, Informix, IBM, Computer Associates, and other vendors were strong and loud. Who can forget the endless benchmark wars and tit-for-tat marketing? What about the colossal exhibits and swarming crowds at major industry gatherings?

Commanding piles of code that had the power to separate the information rich from the information poor, titans of the database era were like rock stars. And the celebrity status was not exclusive to vendor CEOs: Where corporate information centers clustered in New York, Chicago, or in the San Francisco Bay area, top database managers could literally walk across the street and double their pay. Databases were big.

Well, they're a lot bigger now. Our cover story by Richard Winter and Kathy Auerbach analyzes key technology trends based on their unique survey of many of the largest database installations on the planet. Perhaps it comes as no surprise, but a system commanding a few terabytes of data hardly raises an eyebrow today. Since Winter Corp.'s last survey in 2001, "the largest transaction processing database almost doubled," Winter and Auerbach write. "On the decision support system side, the 10th largest database is nearly the size of the largest database in the 2001 program."

With technologies such as radio frequency identification (RFID) coming into the mainstream, we could be on the verge of yet another spurt in size and scope. For organizations that put a high priority on combing through vast amounts of data for predictive analysis and data mining, databases can never be big enough. Content management and XML technologies are bringing increasing structure to the unstructured masses of data — progress that will continue to revise our thinking about what "large" means for databases.

Yet, with the competitive marketplace boiled down to three or four dominant players — IBM, Microsoft, and Oracle, with NCR/Teradata in the mix for data warehouses — and with market growth incremental, eyes are focused elsewhere. The irony is that recent years have featured more technology innovation in databases than what I saw during the heyday.

EXPANDING DATA UNIVERSE

Beyond all that's gone into supporting the incredible scalability requirements discussed in our lead feature, we've also seen the steady incorporation of relational extensions to handle object-oriented applications and unstructured information. Extensions are also bringing higher performance to business intelligence and OLAP functionality. Maturing support for clustered, distributed systems will change the economics of databases, especially if Linux becomes the operating system of choice for such platforms. As database technology becomes cheaper, we will see it embedded everywhere.

Business process integration, Web services, and innovations, such as service-oriented architecture, dominate minds in application development and management. However, competitive advantage and business agility will come through information richness on a scale that few should take for granted. For database professionals, greater fame and fortune could lie ahead.