Amazon CTO: 'We've Just Scratched The Surface'

Sometime in the next few weeks, Amazon.com is scheduled to release Amazon Web Services 4.0, the next version of the electronic retailer's toolset for developing applications that tie into its Web site. It's the next step in Amazon's strategy is to create a "programmable" Web site. Over the past two years, 50,000 developers have downloaded earlier versions of Amazon Web Services. InformationWeek senior editor-at-large John Foley asked Al Vermeulen (pictured right), Amazon's chief technology officer, about how the model works.

InformationWeek: What kind of a technology infrastructure is needed to build a programmable Web site?

Vermeulen: We use the same infrastructure that we use to surface our human-readable Web site, so none of that technology is really that different. Just as we've taken our underlying commerce technology and exposed it to human beings via HTML and the Web, we've taken exactly that same underlying platform and exposed it to programmers using Web services.

We don't go into detail about what our underlying infrastructure technology looks like. In general terms, we have a factory of Web servers. We use a lot of open-source technology, Apache for our Web servers, and underlying that is code written in C++, Java, and Perl that serves up XML for Web services. It's exactly the same technology we use to surface our HTML Web site. What that might mean for companies that already have a Web-site presence [is that] having a Web-services presence is not that big a leap.

InformationWeek: How would you describe Amazon's services-oriented architecture? What are the components?

Vermeulen: There's a host of them, probably 15 or 20, maybe more, and they range from components like personalization and search applications, to fulfillment applications, supply-chain services, and on and on. The basis is, let's think about everything we do at Amazon.com and about how we break it up into individual pieces, smaller pieces. What we try to do is break apart a piece of the business. From a technology point of view, that becomes a service. From an organizational point of view, it becomes an autonomous team with their own mission, and then we work on defining the interface to get to that service. We try to solidify that interface and make it permanent and robust. It's kind of a bottom-up, decentralized way of building your technology, as opposed to a top-down way where you try to make all the technology look like one piece.

InformationWeek: Are you using industry-standard Web-services specifications?

Vermeulen: When we expose our services to the outside world, we use industry-standard protocols. To be quite honest, it's not all that important to us. We're not religious about this at all. We offer two completely different ways to get at our services, depending on what taste developers have and the technology they want to use. Internally, we use some of the industry-standard protocols, but for the most part we do not, and most of the reason for that is historical. We've been building a services-oriented architecture for the last six years, and those protocols were just not mature when we were at the point of building this architecture. And really the industry standards aren't the difficult thing; the difficult thing is in figuring out how to [divide] your application into individual services.

InformationWeek: What are some of the issues associated with this approach in terms of scalability, interoperability, or other things?

Vermeulen: Fortunately, we know an awful lot about scalability that we learned from the school of hard knocks over the last seven or eight years by running our Web site. So we're pretty good at building scalable systems. There's no magic there; it just takes dedicated, smart engineers who know how to build systems that scale. In terms of other challenges, there's certainly a challenge when you build any sort of decentralized organization with keeping the parts in sync with each other. We believe strongly in hiring the very best people and then empowering them, and those people are smart enough to work with each other and make decisions that are coherent with each other. It's not perfect. We do sometimes have issues where it's sometimes difficult to get the pieces of our architecture to work together perfectly, but by and large, it works for us.

InformationWeek: Are there differences in your approach when it comes to dealing with small applications developed for lightly trafficked Web sites on the one hand, and business-to-business connections with large partners on the other?

Vermeulen: It's one of the things that's really changing for us internally with regard to our Web-services offering. When we launched it a couple years ago, it was really a kind of an experiment. We thought it would be interesting to take the functionality of our Web site and make it available to developers. And primarily what we were thinking of were the [Amazon.com] associate Web sites. What's happened is that people have built businesses on our Web services, and these businesses want to run at scale, and it's more important than ever that our Web services be rock solid and absolutely reliable. So that has given us renewed focus internally to focus on the performance and reliability of our Web services.

In terms of other things beyond the technology itself, they do have different kinds of needs. The large outfit will typically have more-sophisticated developers that will want a finer level of granularity access into our services, and the smaller guy's likely to have people working on the site with no real development skills or just some skills with scripting languages and technologies like that. So our developer-relations program has to understand how to interface with each of those two different kinds of teams. As we go forward, we'll get better at providing documentation applicable to each of those development groups.

InformationWeek: Are there things about your experience with this programming model that are applicable to other businesses that are trying to learn how to use Web services?

Vermeulen: What we've really learned is that there are a lot of smart people in the world who don't work at Amazon.com. We've learned that there was even more value than we thought there was going to be in some of the internal pieces that we have and making them accessible to the outside world. People have been very creative in the kind of applications they've built on our Web services. It helps them, and it helps us as well. When we started this program, we had a very deliberate strategy of erring on the side of providing more functionality than we thought would be necessary to developers in the outside world, and that's really paid off. People have used the functionally to do things that we wouldn't have anticipated, as opposed to a strategy of kind of figuring out what you think people are going to build and making the pieces available for what you think they're going to do. My advice would be to go bottom up and expose as much of your technology as you can and let the developers in the outside world innovate. It's just amazing what they'll build.

InformationWeek: Do you think other businesses could do something like what Amazon has done in creating communities of programmers that hit up against their Web sites?

Vermeulen: Absolutely. If you look around the Web right now, there's an enormous amount of information and interesting [processes] that are out there. If it was all exposed via Web services so that developers could integrate that stuff together, people would do incredible things with it. I think that's already starting to happen, and we'll see that accelerate. If businesses want to participate in that, they're going to have to make their underlying technology available via Web services.

InformationWeek: Where do you go from here?

Vermeulen: We're going to keep going on our current direction and accelerate. So we are recruiting like mad in the Web-services organization. We've had success with this program, and we think that we've just scratched the surface, so we're going to go full bore in exposing all of our platform via Web services and making it even easier than it is today for developers, use more outreach programs, encourage more people to build applications, and just keep this whole positive feedback spinning. That's the main direction. The other answer is that even we're not sure. What are the cool things we can do to leverage this position we're in and really help the industry move forward? We're spending a lot of time thinking about that. So, if you have great ideas, send them my way.

InformationWeek: Is there a return-on-investment case in what you're doing?

Vermeulen: For Amazon, there certainly is. We think a lot about feedback effects and about getting as much traffic as we can moving through our platform. How do we increase selection? How to we lower our cost structure? How do we make better tools available for consumers? We think all of these things feed into this cycle of having more shoppers on our platform and more selection, which lets us lower prices further, which brings more shoppers and more selection. We think Web services kind of feeds directly into making that flywheel spin faster. So, yes, for us there's been great ROI, and some of it's even measurable because people build applications using our Web services that drive traffic to our platform. We can't measure how much revenue that generates, but aside from that, we just believe that Web services is an important thing in terms of having better applications for our customers and our sellers, which will encourage more people to use our platform and drive more selection. It's harder for me to speak for other businesses. My gut feeling is that for most businesses out there, there are models that make sense for them to expose their stuff via Web services.

InformationWeek: To what extent does this boil down to an architectural decision around Java or .Net or some other approach?

Vermeulen: One of the great benefits of Web services is that it doesn't matter what technologies developers use. It's very similar to the Web: As long as you follow a few very simple protocols, lots of people can access your technology. To make it very concrete, we did an interesting demo at the Microsoft Professional Developers Conference where we used Microsoft technology and built a very cool application that, underneath the covers, used Amazon Web services. And what's so interesting is we're a Linux shop and use primarily open-source technologies. Microsoft is all Microsoft. And everything fit together great.

Return to main story, New Face Of E-Commerce