The Weather Company Brings Together Forecasting And IoT

Agile Analytics: 11 Ways To Get There

The Weather Company and its project to modernize its data collection, storage, and forecasting platform won recognition in last year's Elite 100, coming in at No. 5.

So it's no big surprise that, a year later, The Weather Company is in the top 5 again as the company continues to build on its previous success. Its expanded ambitions involve its new parent company, IBM, and a plan to apply Watson cognitive computing to the Internet of Things (IoT).

What does all that have to do with weather? "Weather, at the end of the day, is the original big data problem," said CIO and CTO Bryson Koehler.

Koehler joined The Weather Company in July 2012 to update the company's infrastructure of 13 maxed-out data centers and aging apps and turn it into a platform that could be leveraged by external companies. He led the team that created a new cloud-based, cloud-agnostic, data driven infrastructure for predicting the weather and providing API-based delivery of weather-related content.

That project put weather prediction information in the hands of people who need it, and that's a lot of them, because weather affects every person, every industry, and every business in the world.

The Weather Company estimates that weather is perhaps the single largest external factor affecting business performance, to the tune of nearly $1 trillion lost annually in the US alone. Combining weather data with business data can improve decision-making for a wide range of companies across many industries -- from retailers looking to stock the right SKUs and optimize supply chains, to insurance companies that want to advise policyholders about ways to minimize damages in the event of impending severe weather.

Weather, it turns out, is only the first phase for The Weather Company's platform. With the rise of the Internet of Things (IoT), The Weather Company has positioned itself to move well beyond its original charter of predicting weather and helping companies make better decisions based on weather predictions.

After all, weather stations are really IoT devices, Koehler said. They are equipped with multiple sensors for detecting barometric pressure, humidity, temperature, wind speed and direction, and other values.

The Potential of IoT Data

It was the potential of the platform for IoT data that first piqued the interest of IBM, Koehler said. IBM announced plans to acquire The Weather Company in October 2015, saying that it would use the company's platform as the basis of its IoT effort for its cognitive computing business, Watson. The deal closed in January.

The original partnership between IBM and The Weather Company grew out of a conversation at a bar in Las Vegas during a big data event in October 2014. Koehler was a speaker at the conference, and at the end of the day he had drinks with Mac Devine, CTO of the IBM Cloud organization.

Devine told Koehler that IBM wanted to build an IoT capability for its Watson cognitive computing platform, and the company wanted it right away. IBM was trying to move beyond bespoke solutions created anew for every single client and into an API-driven creation of applications for clients.

"I said, 'Well, don't go build that, I've already built it,'" Koehler said he told Devine. "'Why don't you just use our platform?' And he said, 'Well, that's great but isn't that just a weather platform?' And I said 'No!'"

The Weather Company Teams With IBM

That conversation led to a partnership between The Weather Company and IBM, announced in March 2015, that let IBM license use of the data platform The Weather Company had built, Koehler said. The agreement also included The Weather Company moving its platform to IBM SoftLayer, the tech giant's cloud platform. (The Weather Company's data platform operates on both the IBM cloud platform and on AWS today. A spokesperson said the company plans to expand it to run on other cloud infrastructure platforms, too.)

IoT and cloud computing enabled collection of data from more than 100,000 weather sensors and aircraft, from millions of smartphones, buildings, and moving vehicles, IBM noted in its announcement of the partnership with The Weather Company last year. Weather information and IoT information work in similar ways, and that's what is enabling the extension of The Weather Company's modernized platform, originally called SUN (Storage UtilityNetwork) and built to be hosted on any public cloud service.

"We built this very agnostically," Koehler said. "If you think about the weather, the data types that come into weather are all over the place -- lightning data, pollen data, radar data, which is streaming data. We have model data that comes in. We have crowd-sourced user-generated content data where people are doing crowd reports. We have personal weather stations, which send us data, which are really just IoT devices. So we have hundreds of different types of data, hundreds of terabytes a day coming in."

The weather data, storage, and prediction platform that Koehler's team had built was effectively an IoT data platform.

"Yes, we applied the platform to building weather forecasts and powering our mobile applications and our B2B applications," Koehler said. "But fundamentally the work we do to protect and prepare people and businesses for tomorrow is no different than what any business application is trying to do as it looks at vast amounts of data, creates insights off that data, and then helps a business make a smarter decision."

Koehler said the goal is to move from simply predicting what will happen to actually helping businesses use that information to make smarter decisions.

The partnership with IBM, announced in March 2015, soon turned into much more. "Everybody quickly recognized that this would be really powerful if we went deeper," Koehler said. "And because we were talking about something important, we really had to get to a place where all the intellectual property was owned by the same company so we could completely share it."

IBM announced in October 2015 that it would acquire many of the assets of The Weather Company — all the forecasting, analytics, and data assets, and also all the brands and applications other than The Weather Channel television property. The deal included The Weather Company's B2B mobile and cloud-based Web properties, including the professional services arm, WSI; the weather.com website, Weather Underground, which includes the network of personal weather stations; and The Weather Company brand. The Weather Channel TV segment was not included in the purchase, but continues to license weather forecast and analytics from IBM under a long-term contract.

Artificial Intelligence-as-a-Service

When the deal was completed in January 2016, The Weather Company Chairman and CEO David Kenny was named to lead the IBM Watson organization, one of the core businesses in which IBM expects to drive growth for the company in the future. Kenny said then that his goal was to make Watson into a more cohesive product that would offer Artificial Intelligence-as-a-Service.

"What Watson knows today is awesome," said Koehler. "But for Watson to continue adding incremental value to its customers, Watson has to get smarter."

The Weather Company's platform has added new data, architecture, and analytics to the mix. Its platform leverages machine learning and 249 different open source tools, according to Koehler. In addition, it includes proprietary capabilities. Most of the platform was written in Scala, and a few of the technologies it leverages include Cassandra, Spark, Riak, and Redis.

In the months since the acquisition was completed, Koehler's effort has been very much focused on the work required for building out the IoT insights platform for Watson to support internal groups across IBM.

IoT is one of many platforms built on top of the Watson cognitive computing platform, which Koehler explained is designed to have many hundreds, or even thousands of applications built on top of it. "So if you look at Watson IoT or Watson Health, those are Watson applications. And there will be hundreds of those. They are distinct applications solving specific needs in specific industries that have specific use-cases."

That's the idea behind Watson and the concept of Artificial Intelligence-as-a-Service. "When you think about moving from what was once BI and analytics, and now you are talking about insights and cognitive computing capabilities, we want to bring that to life," Koehler said. "Cognitive computing is about taking it to action. It's the ability to enable a business to react in real-time and do something different."

What advice does Koehler have for CIOs looking to bring together data, cognitive computing, and IoT in their organizations? "Be brave. Be bold. Don't spend your time creating a new shade of red, but rather create an entirely new color," he said. "So many IT shops are in need of radical change and transformation that will only come through taking bold steps with high velocity. Clearly this comes with some risks, but I believe that playing it safe is far riskier than taking on some large risks within your organization."