Why NoSQL Equals NoSecurity

If it seems that security is an afterthought in the big data ecosystem, you're right. And that's unfortunate, because attackers go where the data is. Our security surveys consistently show that even conventional structured databases aren't protected as well as they should be. And now we're piling up unstructured data.

We understand how this state of affairs came about. The whole point of NoSQL databases and superfast key-value stores like Redis is to provide rapid, unfettered access to data. The mission statement says nothing about protecting all that data.

This isn't a news flash to security pros, but those charged with managing big data seem unfazed. In our InformationWeek 2012 Big Data Survey of business technology professionals managing a minimum of 10 TB of data, we asked about a dozen management priorities. Robust security came in eighth, selected by just 17% of respondents.

That would be less scary if the No. 1 application driving big data needs at respondents' companies weren't financial transactions.

Clearly, the developers driving the NoSQL bus just don't get it. The only thing we've gotten from years of pushing to secure Hadoop and other big data technologies is integration with authentication frameworks such as Kerberos. Excuse us if we don't swoon with gratitude.

As technologies like Hadoop and NoSQL go mainstream, this situation must be addressed. In 2010, only a handful of companies, notably Foursquare and Craigslist, were heavily into unstructured data, and they didn't deal with sensitive information. But 2011 was a turning point, says Max Schireson, president of 10gen, developer of the NoSQL database MongoDB. "We went from a handful of [employees] to over 100," says Schireson. "We can barely keep up with demand."

10gen's customer list has expanded to include financial services companies such as Intuit, big consumer brands like Disney, and the U.S. intelligence community. InformationWeek's 2012 State of Database Technology Survey (conducted in November) confirms the upward trend. Among 760 respondents, the number with Hadoop in production, running pilots, or investigating it jumped 17 points from August 2010, to 39%. Use of MapReduce and BigTable also rose significantly. The industry most represented in our State of Database Technology poll was government, with healthcare, financial services, and education not far behind--even JPMorgan Chase is using NoSQL technologies to improve fraud detection.

And yet, the NoSQL ecosystem is woefully behind in incorporating even basic security. MongoDB, for example, includes Secure Sockets Layer support only in the commercial offering. There's an outstanding request from January 2010 to add SSL to the open version.

You're thinking, "OK, they must provide good guidance on how to harden the system, right?" Here's what MongoDB's documentation has to say: "The current version of Mongo supports only basic security." What's basic? "It is often valid to run the database in a trusted environment with no in-database security and authentication (much like how one would use, say, memcached)," MongoDB says. "Of course, in such a configuration, one must be sure only trusted machines can access database TCP ports."

So firewalls are now considered sufficient protection for financial data? And it's not just MongoDB. The security page for Redis states that untrusted access to the key-value store should be mediated by a layer implementing access controls lists, validating user input, and deciding what operations to perform against the Redis instance.

We asked a financial services firm that leverages a NoSQL database for trading information where it would put security controls. At the database or application, perhaps? Its answer: the operating system.

So, yes, the NoSQL world has gone mad, and that's because the big data show is being run by developers, not architects or even system administrators. These developers clearly don't realize that 14% of all breaches last year were caused by compromised database servers. That's second only to point-of-sale servers in terms of a single point of weakness and well above the 9% of breaches that involved Web application servers, according to Verizon's Data Breach Incident Report. It seems developers are stuck in 1998, when perimeter security was state of the art.

So do security teams need to go on a (probably doomed) mission to outlaw use of Hadoop, MongoDB, and other NoSQL technologies, at least until they mature? That depends on how willing you are to get your hands dirty.

Strategy: Why NoSQL Equals NoSecurity

Our full report on NoSQL database security is available free with registration.

This report includes 14 pages of analysis along with best practices and hardening tips. What you'll find:

Get This And All Our Reports

If Developers Ran The World

Steve Ballmer had it right: It's all about the developers, and that's the first place to focus efforts to secure unstructured data environments.

Schireson made it clear that security just wasn't part of the MongoDB thought process until recently, when 10gen's customer base expanded from Web 2.0 companies that generally don't store sensitive information to large financial service firms using NoSQL to mine customer data and patterns. Shireson's recommended approach to securing MongoDB installations is to implement an audit system, use SSL, and perform a system architecture review.

That's not bad advice, but the first two points require custom coding, and the third might not help at all, depending on who's doing the review.

Hint: It better not be a developer.

We believe a much more tactical approach must be taken to hardening your NoSQL database infrastructure. First, as an authentication mechanism, most NoSQL systems support Kerberos, which is better than nothing because it lets you use Active Directory or a specially configured MIT Kerberos server for authentication.

Unfortunately, in our experience working with clients that have NoSQL deployments, we've never seen Active Directory in use. We discuss how to do authentication, logging, and encryption right in our full report. For now, let's focus on the difficult job of securing these databases.

Frameworks To The Rescue

If there's one thing coders love it's rapid application development, and that quest for ease of use just might be the savior of big data security.

As discussed, there aren't many security features built into NoSQL databases, so developers are left to write their own. Rapid application development frameworks such as Spring, Lithium, and Ruby On Rails enable developers to quickly interface with NoSQL technologies without having to worry about the complicated installation and database schema configurations that are part and parcel of conventional SQL databases like Oracle and Microsoft SQL.

These frameworks implement the security features we wish were built into NoSQL databases, including authentication, role-based access control, and encryption. For example, the Spring security framework makes more than 20 capabilities available to developers. These frameworks provide a quick, reliable, and usually well-tested set of security features. Best of all, your developers don't need to reinvent the wheel.

Here are the top security controls we recommend developers implement when using a NoSQL back end:

>> Authentication. Unfortunately, even in 2012, most of the NoSQL installations we see have no passwords and allow anyone to access the database. At best, passwords are user-defined. If you can't use a built-in authentication capability within the NoSQL database, make sure you at least use authentication within the framework.

>> Input validation. While NoSQL databases don't normally suffer from the SQL injection issues found in a conventional relational database management system, they can still be injected using JavaScript attacks and string concatenation. Filtering to remove JavaScript, or setting up the NoSQL database to not allow JavaScript within the store at all, will eliminate this attack vector.

>> Data validation. Most NoSQL databases store documents or other objects that can contain dynamic structures. Leveraging the framework to validate data being written to and read from the database can prevent problems, such as when the system converts from one data type to another without the developer realizing it. Data-type conversions can trigger denial-of-service attacks.

>> Role-based access. Store information on which users have access to what data outside the NoSQL database and have the application enforce these roles.

Many developers argue that adding security decreases performance; that's the most common excuse we hear for why NoSQL deployments use no authentication or encryption. However, Owen O'Malley, a Hadoop engineer at Yahoo, says he saw less than a 3% performance hit in Hadoop when additional security features, such as ACLs and authentication, were enabled. That's well worth it, especially compared with the alternative of cleaning up after a successful attack.

What About The OS?

The operating system on which any database runs should be hardened and locked down. Most NoSQL technologies leverage Linux, so there are a variety of options to choose from. When hardening an OS, focus on four areas: users, permissions, services, and logging. Mechanisms such as Bastille Linux or SELinux can help automate Linux hardening, but we recommend you follow a more structured approach, such as those from the Center for Internet Security or the Defense Information Systems Agency's Security Technical Implementation Guide for Linux. These guidelines have been reviewed and tested by thousands of people and are unlikely to cause problems like incompatibility.

It's important to note that when it comes to Hadoop and MongoDB, properly configuring file system permissions is vital. The Hadoop Distributed File System can be securely configured to give only appropriate permissions to users running various jobs. For example, we recommend splitting MapReduce jobs and HDFS users into two groups, so that you have separation of access. HDFS needs to run NameNode, DataNode, and Secondary NameNode, but MapReduce users need to run only the JobTracker and TaskTracker applications. Creating Hadoop groups allows you to set up permissions, a critical part of any system-hardening process. Without the proper permissions, a user could potentially copy the entire Hadoop or MongoDB instance, load it on a new server, and bypass all of your authentication controls; this is also an argument in favor of encryption, as we discuss in our full report.

Finally, don't run these databases as root. We have seen too many instances of this. Create a separate user, and lock down that user so the database has access to only those directories and executables it needs.

Right now, open source NoSQL technologies just aren't ready for the enterprise when it comes to security. Can you make them ready? Sure, but it comes down to resources--do you have people with the right skills? If so and if you're willing to work closely with developers and analyze your organization's risk, you can implement NoSQL technologies securely. Otherwise, there are commercial NoSQL databases such as Vertica and eXist-db that have security controls built in. Just because some well-known Web 2.0 company uses an open source database doesn't mean you should. Their risks, data, and expertise are likely very different from yours.

We're not trying to paint the future of big data and NoSQL as that of a security wasteland. There's precedent for a free-for-all market getting serious under pressure. We saw this happen in the public cloud, as enterprises forced providers to start caring about security controls and privacy. But the fact is that NoSQL technology is by developers, for developers. Unless companies make data protection a priority--and vote with their budget dollars--we don't foresee the NoSQL community suddenly getting security religion.

InformationWeek: Apr. 9, 2012 Issue

Download a free PDF of InformationWeek magazine
(registration required)