About a decade ago, competition in the database market seemed to be winding down. Consolidation had taken out ASK, Tandem, and Informix as independent vendors, and Oracle, IBM, and Microsoft, in that order, were ensconced as the largest commercial relational vendors. MySQL, meanwhile, was growing to be the one prominent option in the open source realm.

The market share picture isn't that different today, yet the database world is alive with competition, evolving features, emerging applications, and promising new possibilities. Whether the database management challenge is data warehousing, data analysis, or transactional processing behind applications, there's new functionality to consider as well as upstart vendors and open source movements that are stimulating the incumbents. That's a good thing; it's boosting performance expectations and preventing even the largest, most entrenched vendors from sitting back, content to collect licensing fees without delivering true breakthrough improvements.

Database vendors shouldn't rest because their customers are under constant pressure.IT organizations are being pressed to do more with less, so they want incumbent databases to require less keep-the-lights-on administration. Beyond these basics, IT groups are also facing new challenges, such as the need to cope with huge data growth and new types of data flowing from digital marketing initiatives, social and mobile interactions, sensors, Web log files, and more.

Bigger, better, faster, and cheaper are all powerful motivators that can get IT leaders' attention--and loosen their purse strings--in this environment, so these themes are the ones you'll find behind today's new product and vendor successes.

Early in the last decade, Netezza burst onto the data warehousing scene, promoting rapid deployment and high scalability by way of a prebuilt data warehouse appliance. Greenplum arrived soon after, offering a database designed to scale out on low-cost, industry-standard commodity x86 hardware. Both vendors exploited massively parallel processing (MPP), an approach proven years earlier by Teradata whereby queries against large volumes of data were spread across scores, if not hundreds, of independent processing nodes. The technology affordably handles data at a whole new scale, answering in seconds queries that might have taken hours on more conventional database deployments.

Netezza and Greenplum arrived just as the Internet, e-commerce, digital marketing, and new mobile and telco services were driving explosive data growth. Aggressive pricing helped them establish beachheads and inspire new competitors. The two upstarts also inspired the incumbents to be more competitive, and Oracle, IBM, and Microsoft eventually countered with preintegrated combinations of hardware and software to speed deployment of well-balanced and optimized data marts and data warehouses. Teradata, meanwhile, rethought its enterprise-data-warehouse-only strategy and came out with a smaller, mart-oriented appliance and purpose-built machines for high-performance analytics and high-scale data archiving.

As the startups quickly gained mindshare--and sales--IBM, Oracle, and Microsoft had to respond. Conventional Oracle and Microsoft SQL Server database deployments were reaching their scalability limits, and IBM was being outflanked on in-database analytic processing by the likes of Netezza and Teradata, which had partnered with SAS and other analytics companies. To maintain their database market leadership positions, the giants had to show cus- tomers they could take them into the future.

To deliver something even more turnkey and appliance-like, Oracle introduced its Exadata Database Machine, which debuted in 2008 on Hewlett-Packard hardware but was switched to Sun servers in the wake of Oracle's mega-acquisition. In that same time frame Microsoft acquired upstart DataAllegro to kickstart the development of what would eventually become the SQL Server Parallel Data Warehouse, released early this year. IBM came out with the prebuilt, preintegrated Smart Analytic System built on DB2 in 2009, and in 2010 it acquired Netezza to advance its in-database analytic processing capabilities.

To read the rest of the article,
Download the Oct. 31, 2011 InformationWeek digital supplement