If it seems that security is an afterthought in the big data ecosystem, you're right. And that's unfortunate, because attackers go where the data is. Our security surveys consistently show that even conventional structured databases aren't protected as well as they should be. And now we're piling up unstructured data.
We understand how this state of affairs came about. The whole point of NoSQL databases and superfast key-value stores like Redis is to provide rapid, unfettered access to data. The mission statement says nothing about protecting all that data.
This isn't a news flash to security pros, but those charged with managing big data seem unfazed. In our InformationWeek 2012 Big Data Survey of business technology professionals managing a minimum of 10 TB of data, we asked about a dozen management priorities. Robust security came in eighth, selected by just 17% of respondents.
That would be less scary if the No. 1 application driving big data needs at respondents' companies weren't financial transactions.
Clearly, the developers driving the NoSQL bus just don't get it. The only thing we've gotten from years of pushing to secure Hadoop and other big data technologies is integration with authentication frameworks such as Kerberos. Excuse us if we don't swoon with gratitude.
As technologies like Hadoop and NoSQL go mainstream, this situation must be addressed. In 2010, only a handful of companies, notably Foursquare and Craigslist, were heavily into unstructured data, and they didn't deal with sensitive information. But 2011 was a turning point, says Max Schireson, president of 10gen, developer of the NoSQL database MongoDB. "We went from a handful of [employees] to over 100," says Schireson. "We can barely keep up with demand."
10gen's customer list has expanded to include financial services companies such as Intuit, big consumer brands like Disney, and the U.S. intelligence community. InformationWeek's 2012 State of Database Technology Survey (conducted in November) confirms the upward trend. Among 760 respondents, the number with Hadoop in production, running pilots, or investigating it jumped 17 points from August 2010, to 39%. Use of MapReduce and BigTable also rose significantly. The industry most represented in our State of Database Technology poll was government, with healthcare, financial services, and education not far behind--even JPMorgan Chase is using NoSQL technologies to improve fraud detection.
And yet, the NoSQL ecosystem is woefully behind in incorporating even basic security. MongoDB, for example, includes Secure Sockets Layer support only in the commercial offering. There's an outstanding request from January 2010 to add SSL to the open version.
You're thinking, "OK, they must provide good guidance on how to harden the system, right?" Here's what MongoDB's documentation has to say: "The current version of Mongo supports only basic security." What's basic? "It is often valid to run the database in a trusted environment with no in-database security and authentication (much like how one would use, say, memcached)," MongoDB says. "Of course, in such a configuration, one must be sure only trusted machines can access database TCP ports."
So firewalls are now considered sufficient protection for financial data? And it's not just MongoDB. The security page for Redis states that untrusted access to the key-value store should be mediated by a layer implementing access controls lists, validating user input, and deciding what operations to perform against the Redis instance.
We asked a financial services firm that leverages a NoSQL database for trading information where it would put security controls. At the database or application, perhaps? Its answer: the operating system.
So, yes, the NoSQL world has gone mad, and that's because the big data show is being run by developers, not architects or even system administrators. These developers clearly don't realize that 14% of all breaches last year were caused by compromised database servers. That's second only to point-of-sale servers in terms of a single point of weakness and well above the 9% of breaches that involved Web application servers, according to Verizon's Data Breach Incident Report. It seems developers are stuck in 1998, when perimeter security was state of the art.
So do security teams need to go on a (probably doomed) mission to outlaw use of Hadoop, MongoDB, and other NoSQL technologies, at least until they mature? That depends on how willing you are to get your hands dirty.