CIA: Cloud Solving Our Petascale Data Problem

Cloud computing can help the Central Intelligence Agency do its job better, faster, cheaper, and even more securely, CIA CTO Gus Hunt said in a keynote address Tuesday at InformationWeek Government's GovCloud event in Washington, D.C.

"Cloud computing has emerged to enable us to deliver capabilities we weren't able to deliver before at a scale and price and agility level we were never able to do before," Hunt said. "I have a petascale problem and need a petascale solution."

That petascale problem Hunt has is data, and lots of it. "The volumes are so big that the wheat to chaff ratio is miniscule and we've got to be able to find this stuff," he said. "This is about being able to correlate data ahead of time, about using the computational capacity of the cloud to see how things are related before you even do a search."

Cloud computing will help him manage that data with technologies that include distributed database management system Apache Cassandra, which was initially developed for Facebook, and cloud data platform MapReduce. Hunt said that cloud computing will help analyze numbers and patterns to help uncover the next attack, much in the same way that a company like PepsiCo might crunch data to determine how likely a consumer is to purchase a bag of Fritos. Big data will help "grow the haystack and magnify the needles," Hunt said.

[ Learn more about analyzing Big Data. View our slideshow on the 12 Top Big Data Analytics Players. ]

The budget is another key driver of the CIA's cloud strategy, Hunt said. As Congress looks to agencies to cut costs, Hunt said that he expects IT to be among the budget items that see the ax. "Big time budget cuts are coming," he said. "We know this, but rather than sticking our heads in the sand, we need to position ourselves so we are ahead of the problem."

Security is a common concern for IT leaders looking to implement cloud computing, but Hunt says that, if done right, cloud computing may actually improve agencies' security postures. "I believe the cloud, potentially, can be more secure than the standard mode of compute," he said.

Much of this improvement comes from the fact that cloud computing requires a dynamic virtual environment, and the CIA can constantly re-image virtual machines, so that computers are always up to date and adversaries never know what is running on which physical computer. "I've created a world in which my workloads are constantly fresh and clean, and I have created a shell game by turning myself into a polymorphic attack service," he said.

The CIA already has private clouds, but they have been developed in "highly specific environments for highly specific workloads," Hunt said. The CIA is now working toward a general purpose private cloud to host a variety of workloads, and working with other members of the intelligence community to determine how to stand up a common set of shared resources that all intelligence agencies could access. One of the CIA's "big bets" for IT, Hunt said, is to serve the CIA by supporting the intelligence community, which means building systems that the rest of the intelligence community can leverage and vice versa.

Hunt ran through a list of attributes for his optimal cloud strategy, including "ruthless" standardization, "rigorous" automation of services, dynamic and elastic commodity computing resources, massive capacity that runs ahead of demand, and a design that's built for speed.

While most of the CIA's cloud computing strategy revolves around private clouds, Hunt sees a role for public cloud computing in the intelligence community. First, public clouds could be used to host unclassified data. Second, and perhaps more important, the CIA is looking to work with vendors to bring their public cloud environments inside the intelligence community firewalls. In that scenario, Microsoft or Amazon might help build an instance of Azure or EC2 that runs inside the CIA rather than on the public cloud. Hunt says that the companies have been receptive to the idea, which would be designed to help the agency reduce costs.