NASA's JPL, No. 3 in the InformationWeek Elite 100, has turned to cloud computing to make real-time analytics available to missions.

Thomas Claburn, Editor at Large, Enterprise Mobility

April 28, 2015

6 Min Read
<p align="left">Curiosity takes a selfie on Mars</p>

NASA's Jet Propulsion Laboratory (JPL) has been putting rockets into the air since World War II, but its involvement with cloud computing goes back only about six years.

JPL is a federally funded research center, run by the California Institute of Technology (Caltech) for space exploration projects, particularly robotic missions. Despite its mission to "dare mighty things," JPL must also be cautious in order to protect lives, complicated equipment, and money while managing missions in extreme environments.

JPL built the Curiosity rover that landed on Mars on Aug. 6, 2012, as part of the Mars Science Laboratory mission. The people who work there deal with some of the most advanced technology on Earth (or anywhere else we know of) but do so in a context where risk must be minimized, because so much can go wrong and the stakes are so high.

For CIO James Rinaldi, the challenge has been making IT more of a platform for innovation across the organization. "The role of IT at JPL has been mixed over the years, and traditionally has been more institutional," he says.

"Over the last several years, we've been able to be more impactful to the real business of the lab, which is the mission work we do."

Tom Soderstrom, chief technology and innovation officer in JPL's Office of the CIO, says Rinaldi uses the term "chaotic architecture" to describe his approach, in which IT helps lead, rather than merely participating. The IT group, he says, has been trying to accelerate innovation without adding risk.

One project based on this approach is called Streams.

Cloud Proves Its Worth

When the Curiosity Rover landed on Mars in 2012, JPL used Amazon’s public cloud computing infrastructure to handle video streamed to Earth. The lab subsequently suggested that Amazon create a cloud that met the government security and regulatory rules it faces, which Amazon did with the introduction of AWS GovCloud. JPL began using that in 2013 to process and protect sensitive data. Its hybrid-cloud infrastructure now spans various clouds -- JPL's private cloud, AWS GovCloud, AWS, and Microsoft Azure, with more on the way.

It proved to be an affordable, functional way to handle the 175 terabytes of video data streamed from Mars, 90% of which was transmitted over the first few hours of the landing.

Streams represented a new attempt to apply this approach to thermal telemetry data from Curiosity on Mars so that the data could be analyzed in real-time, as it was received. Being able to accurately assess and respond to changes in thermal parameters affecting the rover could influence the vehicle's drive time by as much as 40%, according to Soderstrom.

The instruments on Curiosity can only operate within certain temperature ranges, and those ranges aren't the same for all instruments, so it requires some analysis to know what Curiosity can do at any given time on Mars. "Because it is difficult to understand all the parameters for all instruments without interactive access to all the data, we have to be conservative," explains Soderstrom.

JPL operations personnel in the past gathered telemetry data in PowerPoint slides to present to scientists for later analysis. It was a laborious, inefficient process that took weeks and lacked recent data.

JPL developers and operations personnel turned to JPL's Analytics Cloud, a project created in 2013 that's based on Amazon Web Services' GovCloud service and on private cloud infrastructure. In May 2014, they conceived of a project utilizing the Analytics Cloud called Streams.

The team built a Streams prototype in just a few weeks, and implementation began in June. By September, it became available to the Curiosity project. The Curiosity team then could interact with some 200 million telemetry data points in real-time, and it did so in a way that didn't jeopardize critical data.

Soderstrom explains that while JPL's use-case -- base telemetry -- isn't common for other businesses or government agencies, the problem its operations personnel faced -- getting data off old equipment -- is something any business can relate to. The systems in question had been around for a long time. The databases were full and couldn't handle the load. There were long waits and reboots. Shifting to a cloud-based architecture changed all that.

Beyond eliminating the database bottleneck, Streams and the Analytics Cloud proved that innovations could be deployed rapidly.

"It became an innovation factory," says Soderstrom, as the Analytics Cloud became a framework for innovating quickly and trying new IT on real mission data without adding risk.

The Streams team received copies of telemetry data. It was then able to apply data analytics and present the results to Curiosity's operations team in order to show them what Streams might look like if adopted.

The mission environment at JPL has evolved over many years and can't just be tinkered with, explains Soderstrom. But Streams helped overcome institutional caution.

"That's the magic of this," says Soderstrom. "If IT comes in and says 'You have to change the way you do business,' it's a non-starter. Instead, they could keep doing the way they did business. But they could also now do it a better way."

In the end, the better way sold itself. PowerPoint slides and emailed charts lost out to the convenience of the cloud and real-time data availability.

"Having the ability to look at the data a lot of different ways, which is so obvious to us IT guys, is not always so obvious to others," says Rinaldi, who notes that being able to help mission operators has helped build acceptance for cloud infrastructure and trust for IT as a partner.

Faster Analysis

Impressed by the utility of Streams for Curiosity, engineers from the Soil Moisture Active Passive (SMAP) project, also run by JPL, adapted the system for their own use.

Launched in January, SMAP is an Earth satellite mission designed to gather atmospheric and weather data to help predict droughts and global climate change. Though SMAP operates closer to home than Curiosity, many of the technical challenges are similar. Soderstrom says that Streams proved its value immediately. During launch, he says, the system acted up and operations personnel were able to use Streams to see that the data pattern had occurred before, during the test phase. "So it was okay," he says, "it was normal behavior. In five minutes, they were able to answer that question, which otherwise would have taken weeks to do."


Having different types of data -- science data, engineering data, and telemetry data -- in one place where it can be subjected to analysis quickly and easily is a really big deal, says Soderstrom. And it will become more important as technological improvements increase the flow of data. "We're going to get lots and lots and lots more data," he says. "The instruments now on satellites are orders of magnitude higher than they used to be." It's also a big deal for JPL as an organization. These IT successes "change the culture over time," says Soderstrom. Rinaldi points out that they also help with recruitment, because open source code and rapid deployment to the cloud reflect "how the next generation wants to work."

About the Author(s)

Thomas Claburn

Editor at Large, Enterprise Mobility

Thomas Claburn has been writing about business and technology since 1996, for publications such as New Architect, PC Computing, InformationWeek, Salon, Wired, and Ziff Davis Smart Business. Before that, he worked in film and television, having earned a not particularly useful master's degree in film production. He wrote the original treatment for 3DO's Killing Time, a short story that appeared in On Spec, and the screenplay for an independent film called The Hanged Man, which he would later direct. He's the author of a science fiction novel, Reflecting Fires, and a sadly neglected blog, Lot 49. His iPhone game, Blocfall, is available through the iTunes App Store. His wife is a talented jazz singer; he does not sing, which is for the best.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights