Cloud // Cloud Storage
Commentary
11/15/2013
08:00 AM
Rich Wolski
Rich Wolski
Commentary
50%
50%

Why Cloud APIs Don't Matter

Rich Wolski, known for his work at Eucalyptus Systems, argues that architectural features that define how each cloud works are more important than a cloud's APIs.

The power of abstraction

It is the design of the abstractions (the logical function of the cloud services) and not the APIs that result in desirable properties such as scale, resilience, and secure self-service. For instance, in most clouds it is not possible to specify on which specific machine a virtual machine (VM) should be launched. The abstraction says that a VM will exist sometime after the run command is issued (or an error will be reported), but no indication of where the VM is residing will be provided.

The reason for this formulation is so that the cloud can use eventual consistency to achieve scale. Since the user cannot specify where the VM is to run, the cloud can delay the decision (a process called delayed binding) as long as possible. As a result, a specific "slot" for the VM does not need to be reserved (e.g., a lock is avoided) allowing the system to respond to as many requests simultaneously as possible.

Notice that this abstraction property holds no matter what the API's properties might be. For example, the API might be synchronous or asynchronous -- that is, it may be that the user command waits until the VM start request is completely fulfilled or the user is allowed to continue and must check back from time to time (as in the AWS API) to determine when the VM is ready for use.

Note also that it is possible for a bad API to mask the useful properties of a good abstraction. Consider all of the graphical management dashboards that require individual IP addresses to be entered instead of a range. This API does not work at scale due to the possibility of data entry error and the time required.

Thus APIs actually do matter, but they matter for a different reason. The APIs must make the cloud services facile but it is the services themselves and not the APIs that matter when it comes to the question of cloud interoperability.

If only it were that simple

Some of the Cloud cognoscenti have, from time to time, argued about the need for this cloud to support that API in service of cloud interoperability. Randy Bias of CloudScaling and other OpenStack contributors who support his point of view have argued that OpenStack should support the AWS APIs. If only the problem were that simple.

A direct API translation is possible if the underlying abstractions are the same. If they are not, then bolting the same API onto a different abstraction will yield different behavior from the same set of commands. From an application perspective, this has serious consequences. If the APIs look the same but the cloud services yield different results, it is difficult to move an application from one cloud to another and still know what's going to happen at the application level.

Thus, debates about one cloud supplier "adopting an API" from another cloud supplier are usually arguments about whether it's necessary to re-engineer that supplier's underlying abstractions.

So how different are they?

The good news for cloud interoperability is that the core abstractions implemented by different IaaS clouds are similar. This begs the question, "If two clouds with different abstractions are to interoperate, how much effort is required to refactor their internal services before applications do not see a difference?"

I wish I knew. However, until the true source of cloud incompatibility shifts from a discussion of APIs to a discussion of service abstractions, I fear we will not make much progress toward an understanding.

Developers, however, vote with their apps. When enough developers vote for an API and interoperability, a de facto standard emerges, and other clouds have the option of changing to adopt it. Even so, real interoperability will require that their internal service abstractions to be the same, not just the APIs.

Debating the real issues

It is possible for clouds to implement application portability. However, achieving this goal requires more than a debate over cloud APIs. The community needs to develop an understanding of the architectural features that define how each cloud works -- that is, the cloud abstractions -- and to debate which architectural features are both feasible and essential.

Despite the discourse on APIs that has taken place, with its heroes and villains, the real debate over how to make cloud computing a universal platform has not yet begun.

Want to relegate cloud software to edge apps or smaller businesses? No way. Also in the new, all-digital Cloud Software: Where Next? special issue of InformationWeek: The tech industry is rife with over-the-top, groundless predictions and estimates. (Free registration required.)

Previous
2 of 2
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Page 1 / 2   >   >>
J_Brandt
50%
50%
J_Brandt,
User Rank: Ninja
11/24/2013 | 4:49:15 PM
Re: Your services not like Amazon's? Forgettaboutit!.
Ahh thank you both for the clarification and education.
cbabcock
50%
50%
cbabcock,
User Rank: Strategist
11/19/2013 | 5:59:57 PM
Your services not like Amazon's? Forgettaboutit!.
To J_Brandt, yes, both APIs and service "abstractions" matter and both are needed. Wolski's stance reflects a larger debate within the OpenStack community over whether it should "follow" Amazon APIs or diverge and do the best job it can on its own. (Wolski's Eucalyptus is not part of OpenStack.) OpenStack board member Randy Bias charges the OpenStack community in the past has included representatives who irrationally fear and loath Amazon's growing power and won't touch its APIs. That debate has sorted itself out, I think, to say OpenStack will develop its own APIs but will be willing to include the APIs of other clouds. Wolski is saying, among other things, that won't be enough for compatibility. How deep are open source developers willing to go to structure OpenStack services to function like Amazon's? Would Amazon ever come around and say it will follow a shared blueprint for services with OpenStack? Right now, I would bet that thought hasn't occurred to Amazon.
richwolski
100%
0%
richwolski,
User Rank: Apprentice
11/19/2013 | 4:56:12 PM
Re: The Value of Both?
Yes -- you will always need *an* API for the abstractions.  The trouble is that with respect to cloud computing we seem to be obsessing on *the* API.  Which API (or APIs -- there can be several for the same abstractions) is far less important than what the abstractions themselves do.  So when you hear "We'll just bolt this API onto that cloud" such a statement only makes sense of the underlying abstractions are the same.  If not, changing the API won't produce compatibility becasue the underlying abstractions will need to be changed as well.   
J_Brandt
100%
0%
J_Brandt,
User Rank: Ninja
11/19/2013 | 12:20:51 PM
The Value of Both?
I think I am missing something.  Aren't you always going to want APIs in addition to the abstraction?   I may not care which piece of hardware my VM runs, but if I want to read and write to the OS user accounts based on data from my HR system, I'm going to need an API.  The more hybrid your solution, the more need for APIs?
richwolski
100%
0%
richwolski,
User Rank: Apprentice
11/18/2013 | 4:38:36 PM
Re: Cloud Interoperability
Interoperability is key but unless the abstractions are the same, simply having the API be the same doesn't ensure interoperability, regardless of whether the "standard" is open or not.  Back when POSIX was defining standards, for example, there were lots of implementations of the standard that were not compatible but were technically compliant. 
richwolski
50%
50%
richwolski,
User Rank: Apprentice
11/18/2013 | 4:31:20 PM
Re: Why Cloud APIs Don't Matter
Right.  A bad API to a good abstraction is a killer.  However, it is the abstraction that is usually harder to change than the API.  If the API is clumsy or confusing, but the underlying abstraction is valuable, building a new API for the abstraction is usually easier than the other way around.
Li Tan
50%
50%
Li Tan,
User Rank: Ninja
11/18/2013 | 12:19:18 AM
Cloud Interoperability
I agree that the API itself (the detailed function calls, variables, etc.) exposed but the real service behind it. I think the key message here is about cloud interoperability - can we seamlessly port a SaaS application made for Cloud A to Cloud B? No matter how you do it - by using certain kind of cloud API open standard or through proper abstraction on top of API, the application should provide consistent end user experience.
jemison288
50%
50%
jemison288,
User Rank: Ninja
11/16/2013 | 1:27:52 PM
Missing the point by focusing too low
I think this piece fails to hit head-on what is probably the most important aspect of APIs and abstraction layers--specifically, the abstractions above the API layer, not those below or inherent in the API.  Wolski is right that APIs don't matter much, but the reason why they don't matter is that you can abstractions above the APIs.  Increasingly, this is done in one of two ways: using orchestration software (sometimes SaaS) with a built-in abstraction layer to cloud APIs like RightScale/ServiceMesh/enStratius/SaltStack, or heavy use of a PaaS like CloudBees/Appendra/OpenShift that makes whatever integration you're doing with cloud APIs fairly easy to port (as long as you're making sure to use only the basic, reusable units).

So cloud APIs don't matter in the same way that variants of SQL don't matter for most application development: we have abstraction layers (like ORMs) above them.
AndrewBinstock.
IW Pick
100%
0%
AndrewBinstock.,
User Rank: Apprentice
11/15/2013 | 3:57:56 PM
Why Cloud APIs Don't Matter
This has the feel of a strawman chosen b/c it's easy to knock down.

Obviously, if the underlying model is wrong, the API doesn't matter. Is that even controversial?

However, what the author doesn't say, is that if the model is done well but the API is poorly designed, it won't be used. Both the API and the underlying model are important.
jfeldman
100%
0%
jfeldman,
User Rank: Strategist
11/15/2013 | 1:33:03 PM
Re: Are OpenStack, Amazon compatibility possible?
The part that makes me sit up and take notice is the assertion that you could have two of the same API calls to two different services and end up with two different results.  That's the part that makes me think this is more than just semantics.  It would help to have more examples, though.
Page 1 / 2   >   >>
Google in the Enterprise Survey
Google in the Enterprise Survey
There's no doubt Google has made headway into businesses: Just 28 percent discourage or ban use of its productivity ­products, and 69 percent cite Google Apps' good or excellent ­mobility. But progress could still stall: 59 percent of nonusers ­distrust the security of Google's cloud. Its data privacy is an open question, and 37 percent worry about integration.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest, Nov. 10, 2014
Just 30% of respondents to our new survey say their companies are very or extremely effective at identifying critical data and analyzing it to make decisions, down from 42% in 2013. What gives?
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of November 16, 2014.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.