NASA Explores New World Of Open Data - InformationWeek
Government // Open Government
09:06 AM
Connect Directly

NASA Explores New World Of Open Data

NASA has been publishing a lot of data for a long time, but getting it into standard machine-readable formats is still a herculean task.

NASA Spinoffs: 6 Innovations In Health & Medicine
NASA Spinoffs: 6 Innovations In Health & Medicine
(Click image for larger view and slideshow.)

NASA has been an open data operation since the passage of the National Aeronautics and Space Act of 1958, in the very earliest days of the Space Race after Sputnik. The agency has always published untold volumes of scientific data.

Yet the kind of standardized, machine-readable data demanded by the Obama Administration's Open Government Initiative remains a challenge.

"That made more complicated -- or, you might say, made wonderful -- the job we were already doing," NASA open innovation program manager Beth Beck said in an interview. "Big data is NASA -- that's what we have -- but taking all that data and making it machine readable, that's a big job." Most of the data is already digital and readable by some internal applications created by NASA and its network of contractors. The challenge is finding it in a sprawling, decentralized organization and putting it in a form that others can use. Some important data is locked up in the form of PDFs of scientific articles, when a data analyst would much prefer structured XML or even a comma-delimited download of tabular data.

[Seeking truth? Truthy Project Not Orwellian, IT Groups Tell Congress.]

That job rests on Jason Duley, whom Beck introduces as her "data emperor." His real title is lead software developer for the open innovation team. NASA has established the website, which feeds into the centralized catalogue established by the White House. NASA's open data effort has intensified since February or March, when the White House Office of Management and Budget began pushing the open government policy harder, Duley said. A year ago, NASA had 25 published open-data sets. Within a few months, Duley expects to have more than 4,000 available. However, that is still only scratching the service.

"The biggest challenge is finding the data, because NASA is so large, and the field centers all operate like different corporations, and they all have a different governance model," Duley said. There is no guarantee that important data even resides on a NASA server -- sometimes data storage is delegated to a contractor, and there might be no way of knowing that if you weren't part of the project that created it.

Mars data from
Mars data from

"There's not a clear process where someone initiates a workflow to open their data," Duley said. Beginning to create that workflow and a standardized information architecture is one of his current goals. "There should be a notification scheme that will allow us to discover this data, rather than have to hunt for it. I want to delegate responsibility for modeling the data to the people who own it."

The process has to be efficient, Beck said, because the open data initiative is essentially an unfunded mandate, or at least there is very little additional money available to make it work. "It's not our mission to be the data-on-demand agency, but that's one of the mandates. It is dizzying, very hard to keep up with what new mandates are. Even if we put a process in place, the mandates could change."

In particular, the OMB is now asking NASA to identify the top five users of its data from outside government and how they are using that data -- something the agency has never tracked before, or not in a systematic way. The agency is supposed to be able to show "what are they using the data for, what's the financial benefit they get from it, and why do they want the data," she said.

Duley said one possibility for tracking usage more accurately might be to make data available through an application programming interface, rather than straight data downloads. "That would give us traceability into who is using our data, to what extent, and what services they're accessing."

API access to NASA data might also be a way of bringing open data and big data together more, allowing outsiders to access and manipulate the data without necessarily downloading it in bulk. Almost by definition, true big data sets are too large to download easily.

Though Beck's group is working to set up a repository for scientific publications, in most cases it will not host the data itself. Instead,

Next Page

David F. Carr oversees InformationWeek's coverage of government and healthcare IT. He previously led coverage of social business and education technologies and continues to contribute in those areas. He is the editor of Social Collaboration for Dummies (Wiley, Oct. 2013) and ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
David F. Carr
David F. Carr,
User Rank: Author
11/12/2014 | 4:44:31 PM
Re: The data...
I don't know if they're allowed to charge, but I'd think the real benefit would be if they could show the value of what they do and get a little more support from the business community and the taxpayers.
Thomas Claburn
Thomas Claburn,
User Rank: Author
11/12/2014 | 4:21:57 PM
Re: The data...
> I only hope that its effort to track not only downloads but usage yields some income streams that might help fund future exploration.

NASA should be charging for API access to the data and plowing that money back into space exploration.
D. Henschen
D. Henschen,
User Rank: Author
11/12/2014 | 1:57:03 PM
Re: The data...
Nice job, as usual, David on explaining a complicated and nuanced topic. I understand NASA's pain in being forced to make data accessible in the raw without much in the way of money or support to reverse-engineer the methods it previously applied to making data usable. I only hope that its effort to track not only downloads but usage yields some income streams that might help fund future exploration. I know we, as taxpayers, have already funded the development of this data, but that doesn't mean that commercial businesses should have free reign to monetize that data.
User Rank: Ninja
11/12/2014 | 1:09:45 PM
The data...
The data NASA has stored away in vaults and filing cabinets (I can't imagine it's digitised everything yet) must contain some treasure troves of insight into the Cold War. If it needs a digitiser, my hand is firmly raised!
Register for InformationWeek Newsletters
White Papers
Current Issue
Cybersecurity Strategies for the Digital Era
At its core, digital business relies on strong security practices. In addition, leveraging security intelligence and integrating security with operations and developer teams can help organizations push the boundaries of innovation.
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll