Software // Information Management
09:06 AM
Connect Directly

2014 State Of Database Tech: Think Retro

Conventional databases from Microsoft, Oracle, and IBM still dominate the enterprise, say respondents in our newest research. What will it take for NoSQL, DBaaS, and distributed systems to break through?

Read the whole story in the new InformationWeek Tech Digest issue (free registration required).

Nowhere in technology is the chasm between "state of the art" and "actual use" so wide as with databases. Tech journalists and industry pundits feed that gap, expending barrels of virtual ink on innovations, including Hadoop, in-memory systems, and Amazon's Redshift. At publication time, a search for "Hadoop" on Google News yielded just shy of 15,000 results, versus 13,400 for "Oracle database."

Yet despite its online popularity, Hadoop is in production or pilot by only 13% of the 956 respondents to our 2014 State of Database Technology Survey, all of them involved with their organizations' database strategies. Compare that with Microsoft SQL Server (75%) or Oracle (47%). Just 5% use MongoDB, 3% have bought SAP Hana, and 1% use Vertica, to name three databases getting big play in the press. Even FileMaker beats startup darlings Cassandra, Riak, and MariaDB.

And we're not talking about small shops here: 48% of respondents hail from organizations with more than 1,000 employees (23% have more than 10,000), and 37% have at least $100 million in annual revenue.

Today's database landscape isn't just static. It's positively retro. Remember 2004? Facebook had just launched, the iPad wasn't even a twinkle in Steve Jobs' eye, and Gartner's database market share report put IBM (34.1%), Oracle (33.7%), and Microsoft (20%) in the top spots. In our survey, Microsoft, Oracle, and IBM still hold the top spots; we do add MySQL, but that's about it for innovation. The top six databases in use are all relational; you have to go down to the 10th most widely used database in our survey to find NoSQL (MongoDB, at 5%).

And those relational databases from Microsoft, Oracle, and IBM? They're essentially just updated versions of the companies' 2004 offerings.

Oracle gets the lion's share of respondents' database spending, with 46% of those using Oracle devoting more than half of their database budgets to it, followed by SQL Server (34% spend more than half), Access (25%), and DB2 (24%). Yet despite stubbornly high pricing and concerns about using older technology as data variety and volume expand, IBM, Microsoft, and Oracle needn't worry about losing customers fast. Among respondents, 56% of SQL Server shops, 44% of both Oracle and MySQL users, and 29% of DB2 shops plan to increase their use of those databases.

Where's the disruption?

IT stronghold
Cloud computing and shadow IT have shaken up many enterprise tech fiefdoms. Not so in databases. The difference is that cloud and shadow IT are driven by employees and developers getting work done without IT's involvement. Databases, however, are different for three big reasons. First, the constituency that most cares about databases -- database administrators -- is usually part of IT. Second, enterprise application developers don't much care which database sits under the abstraction layer, at least not enough to go to battle. IT is charged with making code live, so again, the database decision remains with IT.

Finally, database hardware choice is insanely important. When you're using one server or a small set of co-located systems to handle all of an application's writes, the hardware's speed and uptime are crucial. Very few respondents' production databases even run on virtual machines, much less hybrid or public cloud. And thus, the status quo sticks.

So do you think you can hold pat for another 10 years?

Maybe. But it's no mystery why venture capitalists, cloud providers, and startups are bullish for NoSQL, and companies like ConAgra are ponying up for expensive in-memory technology for a reason. While conventional relational databases can, theoretically, serve any data store use, they're not always the best choice for today's global, varied, and mobilized workloads. And that's not just because the licensing and hardware demands of conventional relational databases are limiting and expensive. More often, alternatives make sense because modern applications have requirements that simply didn't exist 10 years ago.

For example, Forbes estimates that, for every minute is down, the company loses $66,240. We live in an always-on world, where nightly scheduled maintenance windows are verboten. A high volume of writes, broad geographic distribution, and frequent upgrades are facts of life. New databases such as Mongo and Riak were built from the ground up to run on a distributed architecture and address all of these factors.

Engage with Oracle president Mark Hurd, NFL CIO Michelle McKenna-Doyle, General Motors CIO Randy Mott, Box founder Aaron Levie, UPMC CIO Dan Drawbaugh, GE Power CIO Jim Fowler, and other leaders of the Digital Business movement at the InformationWeek Conference and Elite 100 Awards Ceremony, to be held in conjunction with Interop in Las Vegas, March 31 to April 1, 2014. See the full agenda here.

Joe began his career by winning the 1996 Weird Software Contest with the Mutant Chicken Races and creating the first Windows-based iPod application. Over the past ten years, Joe transitioned from development to systems design and data analysis, creating the first BuildFax ... View Full Bio

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
David F. Carr
David F. Carr,
User Rank: Author
3/11/2014 | 1:09:36 PM
Training issue
Isn't this as much as anything a training and skills issue -- the number of people with training and experience managing SQL databases as opposed to NoSQL ones?
Charlie Babcock
Charlie Babcock,
User Rank: Author
3/10/2014 | 10:16:46 PM
Value is clear but NoSQL must wait
As relational database was born in the mid-1980s, there was huge interest in it, along with constant headlines. IBM purposefully undersold it at the start due to customer disbelief and IBM worries about upending existing mainframe data management. The production systems remained such things as Cullinet IDMS, IBM's IMS and various other heirarchical systems. Lots of education on relilability precedes broad market acceptance, even when the usefulness is perfectly clear.
Lorna Garey
Lorna Garey,
User Rank: Author
3/10/2014 | 2:09:49 PM
Re: Great report, but the market is more dynamic today
At the very least, it does seem that NoSQL is putting some pressure put on the Big 3 database vendors to keep innovating and not raise prices quite as much. Competition is competition.
User Rank: Moderator
3/10/2014 | 10:13:45 AM
Re: Great report, but the market is more dynamic today
The issue is really a "scoreboard" one.  $30 billion RDBMS market; $0 billion NoSQL market.  I agree that the analytical database space has shown more progression, but I really see that as a result of how intractable and expensive the original, traditional EDWs were.  Analytical databases are still niche compared to RDBMSs, and much more reporting and analysis is done out of RDBMSs than analytical databases.
User Rank: Author
3/10/2014 | 10:05:59 AM
Hadoop use case
I'm hearing that some banks see Hadoop as a tool in meeting regulatory requirements. Banks are having to do stress testing often enough that they need the speed of Hadoop to meet the demand.
D. Henschen
D. Henschen,
User Rank: Author
3/10/2014 | 10:05:02 AM
Great report, but the market is more dynamic today
Excellent coverage of all things database. And, yes, the most popular databases of 10 years ago are still market leaders today, but Netezza and Greenplum were just getting the ball rolling on what is now a diverse MPP database market. And the NoSQL and NewSQL markets didn't even exist as we know them today (even though key value stores were kicking around). Maybe it's a matter of a glass-half-empty vs. glass-half-full perspective, but I do see the current database market as disrupted if you consider what's growing and where the trends are leading. 
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest September 24, 2014
Start improving branch office support by tapping public and private cloud resources to boost performance, increase worker productivity, and cut costs.
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.