Big Data // Big Data Analytics
11:13 AM
Connect Directly
Repost This

NASA Polar Ice Flyover: One Bumpy Big Data Project

Indiana University researchers use data feedback in flight to guide NASA's ambitious airborne survey of Earth's polar ice. But bring a barf bag if you come along.

Big Data's Surprising Uses: From Lady Gaga To CIA
Big Data's Surprising Uses: From Lady Gaga To CIA
(click image for larger view and for slideshow)

Earth's polar regions aren't particularly hospitable to computers or humans. Building an advanced data management and storage system -- one that operates on bumpy flights in frigid weather, no less -- can be a formidable task.

Doing so, in fact, is akin to building "a supercomputer that can fly," said Richard Knepper, manager of a campus bridging and research infrastructure team for Indiana University's (IU) Research Technologies, a cyber-infrastructure and service center.

Knepper has a lot of experience in this area. For the past four years, his team at IU has provided IT support for one of several research groups participating in NASA's Operation IceBridge mission, an ongoing effort to help scientists study global climate change by collecting radar data on Earth's polar ice sheets.

[ Analytic proof that business trips take a toll? Read Travel Stress Quantified Using Big Data. ]

The IU team provides data management services for the Center for Remote Sensing of Ice Sheets (CReSIS), a National Science Foundation Science and Technology Center led by the University of Kansas (KU).

In a phone interview with InformationWeek, Knepper discussed the challenges of processing large amounts of data in real time aboard research planes, many of which fly bumpy, 12-hour missions over vast ice sheets covering Antarctica, Chile and Greenland.

The CReSIS team provides radar technology that measures physical interactions of ice sheets in polar regions. IU researchers are responsible for storing and processing multiple terabytes of data collected by these radar systems, both in flight during the missions, and later on the ground. CReSIS "has a computer that turns the analog signals into digital signals, and then we provide a storage solution that captures and verifies (data) as it comes off the instrument," said Knepper.

The data management system provides some processing capability in the plane, as well as additional processing on the ground for more in-depth analysis.

Knepper's team tested a new in-flight data copy system, one that allows real-time data processing and archiving, during a research flight last fall to Antarctica. In addition to saving time and money, the system gave researchers immediate access to their data, allowing them to determine in flight which ice sheets needed closer examination.

Falling prices of solid state drives (SSDs) have helped the IU team improve its processing and storage systems. "What we built last fall, and what we've reconfigured for the spring mission, is something we couldn't have done two or three years ago," Knepper said.

He added: "In the future, data reads are going to go up. We're looking at a field mission next fall that will collect half a petabyte of data. So we have to be prepared to collect and manage 500 terabytes, and have it be accessible to the team while they're working."

IU's system includes three servers, each with 24 solid state drives. SSDs function better than hard drives in harsh environments, particularly when planes are stored outside overnight in subzero temperatures.

The IU system includes a fiber channel array that does use mechanical drives for data backups, however.

An Operation IceBridge mission isn't for the easily nauseated.

"Most people describe it as gut-wrenching. It can be pretty wild," said Knepper. "When you're on one of these airplanes, it's not like a commercial flight. They're doing 30-degree bank turns. They fly in very bad weather sometimes."

One of the aircraft used is a P-3, a propeller-driven plane that "tends to have quite a bit of shock and vibration," he added.

E2 is the only event of its kind, bringing together business and technology leaders across IT, marketing, and other lines of business looking for new ways to evolve their enterprise applications strategy and transform their organizations to achieve business value. Join us June 17-19 for three days of 40+ conference sessions and workshops across eight tracks and discover the latest insights in enterprise social software, big data and analytics, mobility, cloud, SaaS and APIs, UI/UX and more. Register for E2 Conference Boston today and save $200 off Full Event Passes, $100 off Conference, or get a FREE Keynote + Expo Pass!

Comment  | 
Print  | 
More Insights
InformationWeek Elite 100
InformationWeek Elite 100
Our data shows these innovators using digital technology in two key areas: providing better products and cutting costs. Almost half of them expect to introduce a new IT-led product this year, and 46% are using technology to make business processes more efficient.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.