NSA Submits Open Source, Secure Database To Apache

The National Security Agency is moving to open source a secure database technology, Accumulo, that it has been developing internally since 2008.

The spy agency over the weekend submitted the project, constructed of about 200,000 lines of mostly Java code, to the Apache Foundation for incubation.

Fine-grained access control is the primary feature of Accumulo that differs from other similar databases. Accumulo includes cell-level security, which would allow organizations making use of it to let users access only particular fields of the database.

In its submission to the Apache Foundation, the NSA said that it expects that verticals that have a keen interest in privacy, such as the government and healthcare providers, may find the most use for Accumulo. Other features include a storage format that improves compression.

NSA is releasing the project through Apache, rather than directly, because it is heavily reliant on other Apache technologies. The project makes use of the Apache Hadoop software framework and file system for distributed applications, the Apache ZooKeeper configuration service for distributed systems, and the Apache Thrift framework for cross-language software development. It's also similar to the Apache HBase distributed database, and like HBase, was based on Google's BigTable database system.

In its submission, the NSA noted that it has been handling Accumulo like an open source program since its initiation, and that it will release hundreds of pages of documentation as part of the open source move. NSA CIO Lonny Anderson told InformationWeek this spring that the agency was considering open sourcing some of the software it uses internally.

According to the NSA, hundreds of developers are currently using Accumulo. In addition to the NSA, further development will be solicited from MIT Lincoln Labs, Carnegie Mellon University, and others.

This isn't the first time the NSA has publicly acknowledged using Hadoop and like technologies for distributed computing. The agency said as far back as 2009 that it was using Hadoop to help develop a collaborative intelligence gathering system. It's also not NSA's first brush with open source. NSA also developed and released a security-focused branch of Linux known as SE Linux.

Join us for GovCloud 2011, a day-long event where IT professionals in federal, state, and local government will develop a deeper understanding of cloud options. Register now.