Bradford Cross never liked the airport delays, the last minute airline announcements of flight postponements, or the unpredictability of air travel, especially to his home airport of San Francisco, where fog periodically obstructs the runway approaches.

When that happens, planes land one at a time instead of two, and delays cascade through the planned landings and departures. Cross, a former developer of programs for hedge fund trading, found his true calling when he became the head or research and development for FlightCaster.

"I hated the flight delays so bad. I thought it was a great problem to solve," he said during a break in moderating the Big Data track at Cloud Connect 2010 March 18 in Santa Clara, Calif. FlightCaster is a one-year-old Web site, that forecasts, sometimes six hours ahead of the airlines, when flight delays are likely.

Flightcaster is an example of what can be done with the new "cloud-based" systems that manipulate lots of data in the cloud, or on very large server clusters on the Internet. They frequently use NoSQL systems, such as Voldemort, HBase, Apache CouchDb, or Hadoop, and Google's BigTable, which can deal with petabytes of data across hundreds or thousands of servers.

Big Data is something of a misnomer, Cross conceded; FlightCaster deals with data in the terabyte range, like many data warehouses, not petabytes. But it is processing large data sets that are being analyzed in new ways, with Hadoop routines repeatedly sorting, filtering and manipulating the data.

FlightCaster needs to do its analysis work as well as continuously respond to individual inquiries quickly. It also needs to scale up as fast as traffic does when delays start to occur. Relational databases, with their more stringent data handling requirements, would tend to fall behind the task, he said.

FlightCaster is an example of what can be done on a large server cluster to produce useful results propagated freely to thousands of end users. Cross said several companies have negotiated or are negotiating with FlightCaster to make use of its data results.

Users load their airline flight number into FlightCaster, either through an iPhone or Blackberry app or through a browser window, and it analyzes the probability of a delay. The system warns the user if the delay is likely to be less than or more than 60 minutes. If more, that's the type of gap that generates the likelihood a connecting flight will be delayed, and FlightCaster's early notification offers the opportunity to look for alternatives, Cross noted.