Big Data Tackles Classic Question: What's The Weather Forecast?

Predicting the weather more than 10 days in advance has long been more art than science. But a San Diego-based startup says it can predict extreme weather conditions, such as severe winter cold spells and searing summer heat, up to 40 days in advance with a 70% accuracy rate.

EarthRisk Technologies, a two-year-old software firm with 11 full-time employees, says its TempRisk application uses statistical methods to study decades of weather data to uncover key patterns, and then applies these patterns to current atmospheric conditions.

"There are dramatic developments in predictive analytics and big data, and they're being used in very obvious ways--and some not-so-obvious ways," said John Plavan, EarthRisk Technologies CEO, in a phone interview with InformationWeek. "I think we've hit on one of those not-so-obvious ways: Really improved weather forecasting."

Conventional weather modeling techniques tend to break down after 10 days, and have proven ineffective at predicting weather 20 to 40 days in advance, Plavan said. TempRisk's algorithmic approach to long-term forecasting includes daily processing of 1.3 billion calculations and 200 weather patterns across the globe.

"The methods we use are traditionally called analog forecasting methods, which have been used for years and years," said Plavan. "A forecaster will say, 'I think I see the atmosphere setting up like the winter of 1985. And the winter of 1985 was generally a pretty cold winter, so I'm going to use an analog method to (predict) that this winter will be pretty cold too.'"

[ Gartner sees big business for big data. See Big Data Drives Big IT Spending. ]

TempRisk's statistical methodology takes weather forecasting into what the company claims is a new--and more accurate--direction.

TempRisk was created by a software development team led by EarthRisk Technologies' co-founder and president Stephen Bennett, an energy meteorologist who's worked previously at Enron, Citadel Investment Group, and Scripps Institution of Oceanography at the University of California, San Diego.

Working in conjunction with energy traders, who helped manage the project to make sure it had real-world applications, Bennett's team compiled a 6,000-page catalog of data of weather patterns and occurrences from around the world, over the past 60-plus years.

"They ended up with this catalog of relationships that they found really promising," said Plavan. For example, "when the atmosphere looks like a certain combination of patterns, 30 days later there's an elevated possibility of an extreme cold event in the eastern U.S."

Plavan, whose varied entrepreneurial background includes stints in the venture capital, real estate, and petroleum industries, co-founded EarthRisk Technologies with Bennett in 2010.

"We built the prototype software interface product around the research output, and tested it in the winter of 2010-11 with three energy trading development partners," said Plavan.

TempRisk's current users include energy traders in four countries, as well as some electric power generation utilities.

"Our primary client is the energy trader, somebody who's looking at the long term--and long term to us is defined as anything past the traditional forecast model," Plavan said. "Anything past about 10 days, they're using our product to inform energy trades, whether they're for power or natural gas."

Plavan believes TempRisk has potential uses beyond energy trading, however.

"There's a need for storm prediction--seeing flood and drought risks," he said. "And our power-generation customers are looking for better insight into long lead times for wind and solar, which are having a better impact on the pricing for energy."

Plavan added: "Our initial market is energy traders because, frankly, there's a huge need for that, and there's lots of dollar value at stake there. And they can fund our development."

TempRisk is available for 10 geographical areas: eight regions in the United States, and one each for Europe and Asia Pacific.

At this hands-on Wall Street & Technology Virtual Event, Big Data On Wall Street, experts and solution providers will offer detailed insight into how risk management, financial reporting, trading analytics and financial modeling, along with a host of other opportunities, can all benefit from applying big data techniques and technologies to business processes. When you register, you will gain access to live and on-demand webcast presentations, as well as virtual booths packed with free resources. It happens Nov. 1.