Cloud // Cloud Storage
Commentary
11/15/2013
08:00 AM
Rich Wolski
Rich Wolski
Commentary
Connect Directly
RSS
E-Mail
50%
50%

Why Cloud APIs Don't Matter

Rich Wolski, known for his work at Eucalyptus Systems, argues that architectural features that define how each cloud works are more important than a cloud's APIs.

Editor's note: This contributor argues that the services behind a cloud API are more important than the API itself. The author founded the open-source project that created APIs and services to match Amazon's. You may not agree, but in the end, he says, it is the success of the underlying services in attracting developers, who create applications to use them, that determines the real value of APIs. His firm, Eucalyptus Systems, is based on the same premise. Its software for private clouds generates services/APIs that closely mimic Amazon's.

There remains a healthy debate over the importance of APIs in the modern world of Web services and cloud computing. A while back, my peers Ben Black of Microsoft, Adrian Cole of Netflix, James Watters of Pivotal, and I were bantering back and forth on the subject. Eucalyptus CEO Marten Mickos does interviews about the subject in between debates with Cloudscaling founder and CTO Randy Bias. My computer science research group at the University of California, Santa Barbara, is even studying API governance as a new challenge facing modern IT managers.

With all of the importance that APIs seem to have garnered, I wish to make a controversial statement that I hope stimulates a healthy repartee among InformationWeek readers:

APIs do not matter, abstractions do.

Put another way, the API is simply the way to "talk" to software that implements a specific functional part of a cloud service or business. To make this functionality easier to use, faster to develop, and more robust, the software implements its functionality in terms of a logical abstraction.

For example, it is easier to talk SQL to a database system and get it to do what we want than it would be to talk directly to the CPU, disks, and network connections that underlie the database and tell them each in turn what we're trying to do. Thus a database is an abstraction that hides the complexity of a physical system. The database API lets applications manipulate this abstract system, but it's the quality of the database abstraction, not the quality of the API, that defines the reliability and benefit of the resulting services. The same is true for cloud services today.

[ A smarter way to run apps in the cloud? Read Containers Add New Efficiency To Cloud Computing. ]

When we started the Eucalyptus project, we knew we wanted Eucalyptus to be compatible with Amazon's AWS, but we didn't spend a lot of time initially trying to understand the APIs. In fact, at the time, AWS supported three different APIs -- REST, SOAP, and command line -- for exactly the same set of cloud services. That is, we focused on understanding the set of cloud abstractions we had to implement as one set of services and then built three different APIs for that set.

Not convinced that APIs and abstractions are different and that abstractions are what matter? When we started Eucalyptus as a company, several competitors claimed that they too supported the Amazon APIs. They had, in fact, read the AWS documentation and implemented the REST API calls that Amazon so carefully specifies. The only problem was that when a customer's application made these calls to AWS and then to one of these other competitor systems, the results that came back were different. That is, there were lots of implementations of the same API that ultimately worked differently, depending on how cloud service abstractions of the particular cloud were implemented.

So why should anyone care?

Previous
1 of 2
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Page 1 / 2   >   >>
J_Brandt
50%
50%
J_Brandt,
User Rank: Ninja
11/24/2013 | 4:49:15 PM
Re: Your services not like Amazon's? Forgettaboutit!.
Ahh thank you both for the clarification and education.
cbabcock
50%
50%
cbabcock,
User Rank: Strategist
11/19/2013 | 5:59:57 PM
Your services not like Amazon's? Forgettaboutit!.
To J_Brandt, yes, both APIs and service "abstractions" matter and both are needed. Wolski's stance reflects a larger debate within the OpenStack community over whether it should "follow" Amazon APIs or diverge and do the best job it can on its own. (Wolski's Eucalyptus is not part of OpenStack.) OpenStack board member Randy Bias charges the OpenStack community in the past has included representatives who irrationally fear and loath Amazon's growing power and won't touch its APIs. That debate has sorted itself out, I think, to say OpenStack will develop its own APIs but will be willing to include the APIs of other clouds. Wolski is saying, among other things, that won't be enough for compatibility. How deep are open source developers willing to go to structure OpenStack services to function like Amazon's? Would Amazon ever come around and say it will follow a shared blueprint for services with OpenStack? Right now, I would bet that thought hasn't occurred to Amazon.
richwolski
100%
0%
richwolski,
User Rank: Apprentice
11/19/2013 | 4:56:12 PM
Re: The Value of Both?
Yes -- you will always need *an* API for the abstractions.  The trouble is that with respect to cloud computing we seem to be obsessing on *the* API.  Which API (or APIs -- there can be several for the same abstractions) is far less important than what the abstractions themselves do.  So when you hear "We'll just bolt this API onto that cloud" such a statement only makes sense of the underlying abstractions are the same.  If not, changing the API won't produce compatibility becasue the underlying abstractions will need to be changed as well.   
J_Brandt
100%
0%
J_Brandt,
User Rank: Ninja
11/19/2013 | 12:20:51 PM
The Value of Both?
I think I am missing something.  Aren't you always going to want APIs in addition to the abstraction?   I may not care which piece of hardware my VM runs, but if I want to read and write to the OS user accounts based on data from my HR system, I'm going to need an API.  The more hybrid your solution, the more need for APIs?
richwolski
100%
0%
richwolski,
User Rank: Apprentice
11/18/2013 | 4:38:36 PM
Re: Cloud Interoperability
Interoperability is key but unless the abstractions are the same, simply having the API be the same doesn't ensure interoperability, regardless of whether the "standard" is open or not.  Back when POSIX was defining standards, for example, there were lots of implementations of the standard that were not compatible but were technically compliant. 
richwolski
50%
50%
richwolski,
User Rank: Apprentice
11/18/2013 | 4:31:20 PM
Re: Why Cloud APIs Don't Matter
Right.  A bad API to a good abstraction is a killer.  However, it is the abstraction that is usually harder to change than the API.  If the API is clumsy or confusing, but the underlying abstraction is valuable, building a new API for the abstraction is usually easier than the other way around.
Li Tan
50%
50%
Li Tan,
User Rank: Ninja
11/18/2013 | 12:19:18 AM
Cloud Interoperability
I agree that the API itself (the detailed function calls, variables, etc.) exposed but the real service behind it. I think the key message here is about cloud interoperability - can we seamlessly port a SaaS application made for Cloud A to Cloud B? No matter how you do it - by using certain kind of cloud API open standard or through proper abstraction on top of API, the application should provide consistent end user experience.
jemison288
50%
50%
jemison288,
User Rank: Moderator
11/16/2013 | 1:27:52 PM
Missing the point by focusing too low
I think this piece fails to hit head-on what is probably the most important aspect of APIs and abstraction layers--specifically, the abstractions above the API layer, not those below or inherent in the API.  Wolski is right that APIs don't matter much, but the reason why they don't matter is that you can abstractions above the APIs.  Increasingly, this is done in one of two ways: using orchestration software (sometimes SaaS) with a built-in abstraction layer to cloud APIs like RightScale/ServiceMesh/enStratius/SaltStack, or heavy use of a PaaS like CloudBees/Appendra/OpenShift that makes whatever integration you're doing with cloud APIs fairly easy to port (as long as you're making sure to use only the basic, reusable units).

So cloud APIs don't matter in the same way that variants of SQL don't matter for most application development: we have abstraction layers (like ORMs) above them.
AndrewBinstock.
IW Pick
100%
0%
AndrewBinstock.,
User Rank: Apprentice
11/15/2013 | 3:57:56 PM
Why Cloud APIs Don't Matter
This has the feel of a strawman chosen b/c it's easy to knock down.

Obviously, if the underlying model is wrong, the API doesn't matter. Is that even controversial?

However, what the author doesn't say, is that if the model is done well but the API is poorly designed, it won't be used. Both the API and the underlying model are important.
jfeldman
100%
0%
jfeldman,
User Rank: Strategist
11/15/2013 | 1:33:03 PM
Re: Are OpenStack, Amazon compatibility possible?
The part that makes me sit up and take notice is the assertion that you could have two of the same API calls to two different services and end up with two different results.  That's the part that makes me think this is more than just semantics.  It would help to have more examples, though.
Page 1 / 2   >   >>
Google in the Enterprise Survey
Google in the Enterprise Survey
There's no doubt Google has made headway into businesses: Just 28 percent discourage or ban use of its productivity ­products, and 69 percent cite Google Apps' good or excellent ­mobility. But progress could still stall: 59 percent of nonusers ­distrust the security of Google's cloud. Its data privacy is an open question, and 37 percent worry about integration.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - July 22, 2014
Sophisticated attacks demand real-time risk management and continuous monitoring. Here's how federal agencies are meeting that challenge.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
A UBM Tech Radio episode on the changing economics of Flash storage used in data tiering -- sponsored by Dell.
Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.