Erik Carlin, Rackspace director of products, said tests have been consistent over time because "those free cycles [are] available most of the time." If Rackspace needed to improve its profit margins, however, one way to do so would be to stack more customers on the same number of servers.
Other Measurements Matter
Even if IT departments manage to get comfortable with CPU, RAM, and storage comparisons, there are other elements that will still have a big impact on cloud performance versus its cost.
Speed of data retrieval is one variable. Jason Read, co-founder of the cloud metrics compiling startup, CloudHarmony.com, offers the example of newcomer cloud vendor Softlayer offering pricing competitive with Amazon, but that's a bit slower because of "1 Gbps SAN off instance storage," Read says, because it doesn't have local disk on the server where the CPU is and relies on a connection to where disk is located on the network.
There also can be different billing cycles. Hosting.com charges by the month, not by the hour. Its basic Linux server is $145, plus $20 for storage. The rough equivalent from Amazon would be its medium Windows server at 23 cents an hour or $165.60 in a 30-day month, but the Hosting.com server is running CentOS Linux, a Red Hat knockoff. It's not clear from the website if the Hosting.com virtual CPU is equivalent to Amazon's medium class server with two ECUs.
And of course, there are different features vendors try to emphasize. Google App Engine offers a basic virtual server for 8 cents an hour, but it also comes with a package of development services and tools for producing software that deploys on App Engine--a model referred to as platform-as-a-service. Microsoft also promotes its Azure service's development platform, and Hewlett-Packard, which is just entering the cloud market, likewise emphasizes the range of development and software services it offers as a platform along with infrastructure.
When it comes to determining the value to IT, it looks like would-be cloud customer will have to make some guesstimates in comparing what these less easily matched up factors are worth. "It's unfortunate," said Cloudscaling's Bias, "but it's still the very early days of cloud computing. We haven't seen any real, credible threats to Amazon."
The reality is that not many companies have moved large-scale enterprise IT workloads into infrastructure as a service. As IT considers doing that, more will tackle the kind of head-to-head, numbers-driven bakeoffs of cloud providers discussed here. If they do that more, you might start to see a thinning out providers of infrastructure as a service, which today is a large and still growing field. What's likely to happen is more companies--like HP--will enter emphasizing more than raw computing, talking up related services such as development tools. Likewise, we'll see cloud infrastructure aimed at specific industry needs.
Even if the day arrives that there is a shared standard for the virtual CPU, the customer will still need a sharp pencil to calculate which service is the right one for a given company. For now, and likely for a while to come, no one seems eager to make head-to-head comparisons easy.
Private clouds are more than a trendy buzzword--they represent Virtualization 2.0. For IT organizations willing to dispense with traditional application hosting models, a plethora of pure cloud software options beckons. Our Understanding Private Cloud Stacks report explains what's available. (Free registration required.)