Cloud ROI: Why It's Still Hard To Measure

Why does calculating cloud ROI remain so tough? Examine the tactics used by two cloud innovators, GE and Airbnb -- plus our exclusive survey.

turbines, appliances, and financial services. As GE moves to a cloud infrastructure, Drumgoole knows he can't force one way of doing things on each business.

"Our view is, we build the roads," he says, including private and public cloud infrastructure. "The businesses units have to decide what they want to drive on them."

GE spends "millions" a year on public cloud services, Drumgoole says, in part because it wants to understand that architecture as it looks to design its own future data centers with cloudlike characteristics. GE plans to reduce its 34 data centers to "less than five" in the next few years.

But GE's calculations for using the public cloud are far more complicated than "which is cheaper, public cloud or in-house?" One factor is whether the data that the application uses can be stored safely in the cloud. For sensitive personal and transaction data, "Amazon can't handle that today," Drumgoole says. "Someday, no problem, but not today."


Which raises the question: Which metrics does GE use to decide on a public cloud service vendor? It uses an instance base model -- a virtual machine with a certain amount of CPU, memory, and storage -- to compare services, but Drumgoole says that's only part of the evaluation. There's also the question of whether the supplier provides solid state disks, the rate of input/output operations per second (IOPS) it supports, and the size of the network pipe running application-to-application or application-to-storage.

[Read related stories: Cloud ROI Easy Win: Disaster Recovery and Milliman Strives For ROI With Azure System.]

Each supplier defines a virtual instance a little differently. When it comes to IOPS and networking, the variances are even greater, Drumgoole says. Cloud ROI comparisons are difficult, but GE, with its far-flung global operations, wants many vendors. "We would like to see more players," Drumgoole says. "We're not as U.S.-centric as some companies."

GE takes an application-centric approach. With so many legacy applications, its challenge is less about choosing one cloud vendor and more about rearchitecting its software for future cloud use. Drumgoole has no interest in moving legacy applications as is into the cloud. He wants to build more generalized services that will reduce the total number of applications he must maintain, cut the number of technologies he's dealing with, and radically simplify the environments for which he's developing future applications.

Drumgoole notes that getting the first 100 applications into the cloud was painstaking. Some of the 9,000 will be rewritten; some will be eliminated as new services are brought online. Migrating many of the remainder will go faster, he says, but it still won't be easy. Most companies are taking the long view in evaluating the ROI of cloud projects, our survey finds. Just over half of respondents at organizations likely to evaluate ROI look at a three- to five-year period, while 30% consider two years or less.

By the time GE is down to just five data centers, it will have selected a standard cloud server design after "a vigorous intellectual debate going on right now," says Drumgoole. That server design will be built by an independent hardware manufacturer, in much the same way that Fidelity and Goldman Sachs are using hardware designs shared through the Facebook-backed Open Compute Project.


GE, like Airbnb, doesn't see cost savings as a primary driver of its migration to cloud services. Likewise, only one in five respondents at organizations likely to evaluate cloud ROI say it's the only or most important factor. Still, 70% say it's important, while just 10% say it's a minor factor or not at all a factor.

Unlike Airbnb, GE knows that some of its resources will be in its own private cloud data centers as well as in public clouds.

Most big companies will live in this hybrid private-public cloud environment for years to come, but it's a difficult path to navigate, given the pricing complexities. GE wants to build a standard set of services and a private cloud environment in which they will run. In Drumgoole's view, IT in the future will tell each unit, "Here's a machine-readable service catalogue" and leave it to each unit to make use of it.

GE's approach leaves some elements of the cloud ROI assessment to be handled down the road by the company's line-of-business managers, as they apply a range of cloud metrics to their specific needs -- a similar scenario most companies will face.

Read the entire Aug. 4 issue of InformationWeek --
no registration required.