Amazon Cloud Turns 10: How It's Changed

10 Startups Aiming To Change The Way You Work

Amazon Web Services emerged from being mere infrastructure for online bookselling on March 13, 2006, 10 years ago. That was the day the first AWS cloud service was announced, Simple Storage Service or S3. Quite a bit has changed since then.

Amazon has not merely built an expanding universe out of its online e-commerce system. (Is there anything it doesn't sell today?) It's also made a profit center out of the AWS infrastructure that supports it. Today's cloud isn't a perfect match for the early book-selling Amazon. On the contrary, it was born from the redesigned and revamped infrastructure that CTO Werner Vogels was convinced he could sell as a service.

"There is no compression algorithm for experience," wrote Vogels in an All Things Distributed blog post on March 11, 10 Lessons From 10 Years of Amazon Web Services. Amazon leads the cloud market, in part because it gained the experience needed to design cloud infrastructure before launching its first service.

To do so, it needed to learn how to build systems that could be repaired while they were running and evolve with the times without being taken down for a revamp. In his first lesson learned ("build evolvable systems"), Vogels quoted Amazon distinguished engineer Marvin Theimer as saying Amazon started out "as a single engine Cessna plane, but over time the plane was upgraded to a 737, then a group of 747s, all the way to the large fleet of Airbus 380s that it is now. All the while, we were refueling in midair."

That's a lot easier said than done. It probably hasn't been done before at the scale at which Amazon is operating.

Learn to integrate the cloud into legacy systems and new initiatives. Attend the Cloud Connect Track at Interop Las Vegas, May 2-6. Register now!

His second lesson -- expect the unexpected -- may have something to do with his long survival as CTO: expect systems to fail. It's fine to assume many failures can be anticipated and that good engineering will prevent them, but "many more are unknown at design and build time," he wrote. This sense of humility was essential to the early cloud's success. Surviving failures was a necessary, built-in goal.

Who could have anticipated the spectacular three-day Easter weekend failure in 2011, when a trunkline network was unplugged by a technician in the middle of the night, and then reconnected to a backup network that couldn't take the traffic? The oldest source of failure, human error, had struck again. The data streams couldn't reach their destinations, and thousands of formerly accounted for data sets had disappeared from view.

That set off a "re-mirroring storm," as thousands of systems realized they'd lost a copy of their data and simultaneously set about creating another backup copy. It was an automated safeguard against the failure of a single system, suddenly activated on a cloud scale.

Systems that were trying to copy their data were temporarily tied up. Systems that were theoretically still running found there was no bandwidth left for them to reach their data. They didn't go down; they just couldn't do anything.

Though not part of Vogel's blog, it's worth mentioning that a similar example hit Microsoft's Azure on Leap Day in 2012, a day Azure had prepared for -- except for the certificate-issuing server, which couldn't issue certificates with the correct date. That lead to virtual machines failing to initiate, and three failed tries leading to an automated migration of virtual machines away from what the Azure governing system presumed was failing hardware.

By migrating virtual machines that couldn't be started, Azure spread the problem around until a major slowdown, and in some cases, a stoppage, occurred. Again, software designed to prevent failures had inadvertently generated one.

His third lesson, which Vogels termed "primitives, not frameworks," was that Amazon realized early on that the way customers would use the cloud would evolve alongside the cloud's development. In other words, the provider should offer basic technology services and see what the customer does with them, instead of providing a highly engineered product that assumes how the customer will use it.

As customers came to the cloud to sidestep IT, "they started to develop systems with new and interesting usage patterns that no one had ever seen before," recalled Vogels.

Lesson 4: The only way a cloud service supplier can offer cost-effective cloud services is to automate its operations. "Automate the management as much as possible," advised Vogels.

In his fifth lesson learned, Vogels said, "APIs are forever," so they have to be done right the first time. They are the front door to any cloud's services. Once they are composed, third-party programming will adopt them, so they can't be changed without causing those programs to break.

Amazon first learned that lesson with its retail operations, so it didn't need to relearn it for AWS. "We knew … we'd only have one chance to get it right," he wrote.

For his other lessons learned, check out Vogel's blog.

These are sound conclusions, but they scarcely sum up what Amazon has accomplished or what it's become over the past decade. It should rightly be called the first decade of digital business, and Amazon, as AWS's parent company, should be considered its leading practitioner. Amazon has illustrated that a sound digital commerce platform -- and an ability to digitize services related to that platform -- is the only choice for a business platform of the future.

Having automated online purchasing, Amazon is well down the road to digitizing distribution and fulfillment. It's created a constituency of online buyers who pay $99 a year for the privilege of spending money with Amazon, and then get free deliveries and other benefits. This is a prized group of consumers in the new economy.

To ensure rapid deliveries, Amazon has signed an agreement to lease 20 Boeing 767s from Air Transport Services Group, according to this report, and will substitute its own package hauling service for some currently used FedEx and UPS delivery services.

[Want to see how Amazon is moving into package delivery overseas? Read Amazon Moves Suggest Package Delivery Is on Its Mind.]

It also has its own distribution centers and same-day truck delivery systems so that it can deliver fresh vegetables to your door.

Amazon is now in the business of creating studio entertainment content to compete with HBO and Netflix. It is trying to find just the right device that captures consumers' loyalty and ties them more closely to Amazon, such as the Kindle Fire and Echo.

In effect, it's inventing new services as it goes along to load on top of those it's already got running. It's a big company, innovating like a small one, and daring Wal-Mart, Microsoft, and anybody else to keep up with it.

The one thing it hasn't done is report big profits. But when there's no one else who can exploit the new cloud services as effectively as it can, Amazon doesn't need to spend all day worrying about profitability. It keeps its universe expanding, and one day, with all those efficient services, the profits are sure to follow. Meanwhile, it's inventing what the new economy will look like, celebrating its tenth anniversary, and spending little time looking back.