IoT
IoT
Cloud // Platform as a Service
News
3/18/2016
08:06 AM
Connect Directly
Twitter
RSS
E-Mail
50%
50%
RELATED EVENTS
Core System Testing: How to Achieve Success
Oct 06, 2016
Property and Casualty Insurers have been investing in modernizing their core systems to provide fl ...Read More>>

Amazon Cloud Turns 10: How It's Changed

Could anyone have anticipated the size of the digital economy that would grow around AWS's S3 storage service when it launched a decade ago?

10 Startups Aiming To Change The Way You Work
10 Startups Aiming To Change The Way You Work
(Click image for larger view and slideshow.)

Amazon Web Services emerged from being mere infrastructure for online bookselling on March 13, 2006, 10 years ago. That was the day the first AWS cloud service was announced, Simple Storage Service or S3. Quite a bit has changed since then.

Amazon has not merely built an expanding universe out of its online e-commerce system. (Is there anything it doesn't sell today?) It's also made a profit center out of the AWS infrastructure that supports it. Today's cloud isn't a perfect match for the early book-selling Amazon. On the contrary, it was born from the redesigned and revamped infrastructure that CTO Werner Vogels was convinced he could sell as a service.

"There is no compression algorithm for experience," wrote Vogels in an All Things Distributed blog post on March 11, 10 Lessons From 10 Years of Amazon Web Services. Amazon leads the cloud market, in part because it gained the experience needed to design cloud infrastructure before launching its first service.

To do so, it needed to learn how to build systems that could be repaired while they were running and evolve with the times without being taken down for a revamp. In his first lesson learned ("build evolvable systems"), Vogels quoted Amazon distinguished engineer Marvin Theimer as saying Amazon started out "as a single engine Cessna plane, but over time the plane was upgraded to a 737, then a group of 747s, all the way to the large fleet of Airbus 380s that it is now. All the while, we were refueling in midair."

That's a lot easier said than done. It probably hasn't been done before at the scale at which Amazon is operating.

Learn to integrate the cloud into legacy systems and new initiatives. Attend the Cloud Connect Track at Interop Las Vegas, May 2-6. Register now!

His second lesson -- expect the unexpected -- may have something to do with his long survival as CTO: expect systems to fail. It's fine to assume many failures can be anticipated and that good engineering will prevent them, but "many more are unknown at design and build time," he wrote. This sense of humility was essential to the early cloud's success. Surviving failures was a necessary, built-in goal.

Who could have anticipated the spectacular three-day Easter weekend failure in 2011, when a trunkline network was unplugged by a technician in the middle of the night, and then reconnected to a backup network that couldn't take the traffic? The oldest source of failure, human error, had struck again. The data streams couldn't reach their destinations, and thousands of formerly accounted for data sets had disappeared from view.

That set off a "re-mirroring storm," as thousands of systems realized they'd lost a copy of their data and simultaneously set about creating another backup copy. It was an automated safeguard against the failure of a single system, suddenly activated on a cloud scale.

Systems that were trying to copy their data were temporarily tied up. Systems that were theoretically still running found there was no bandwidth left for them to reach their data. They didn't go down; they just couldn't do anything.

Though not part of Vogel's blog, it's worth mentioning that a similar example hit Microsoft's Azure on Leap Day in 2012, a day Azure had prepared for -- except for the certificate-issuing server, which couldn't issue certificates with the correct date. That lead to virtual machines failing to initiate, and three failed tries leading to an automated migration of virtual machines away from what the Azure governing system presumed was failing hardware.

By migrating virtual machines that couldn't be started, Azure spread the problem around until a major slowdown, and in some cases, a stoppage, occurred. Again, software designed to prevent failures had inadvertently generated one.

Amazon's first headquarters in Seattle.
(Image: Amazon)

Amazon's first headquarters in Seattle.

(Image: Amazon)

His third lesson, which Vogels termed "primitives, not frameworks," was that Amazon realized early on that the way customers would use the cloud would evolve alongside the cloud's development. In other words, the provider should offer basic technology services and see what the customer does with them, instead of providing a highly engineered product that assumes how the customer will use it.

As customers came to the cloud to sidestep IT, "they started to develop systems with new and interesting usage patterns that no one had ever seen before," recalled Vogels.

Lesson 4: The only way a cloud service supplier can offer cost-effective cloud services is to automate its operations. "Automate the management as much as possible," advised Vogels.

In his fifth lesson learned, Vogels said, "APIs are forever," so they have to be done right the first time. They are the front door to any cloud's services. Once they are composed, third-party programming will adopt them, so they can't be changed without causing those programs to break.

Amazon first learned that lesson with its retail operations, so it didn't need to relearn it for AWS. "We knew … we'd only have one chance to get it right," he wrote.

For his other lessons learned, check out Vogel's blog.

These are sound conclusions, but they scarcely sum up what Amazon has accomplished or what it's become over the past decade. It should rightly be called the first decade of digital business, and Amazon, as AWS's parent company, should be considered its leading practitioner. Amazon has illustrated that a sound digital commerce platform -- and an ability to digitize services related to that platform -- is the only choice for a business platform of the future.

Having automated online purchasing, Amazon is well down the road to digitizing distribution and fulfillment. It's created a constituency of online buyers who pay $99 a year for the privilege of spending money with Amazon, and then get free deliveries and other benefits. This is a prized group of consumers in the new economy.

To ensure rapid deliveries, Amazon has signed an agreement to lease 20 Boeing 767s from Air Transport Services Group, according to this report, and will substitute its own package hauling service for some currently used FedEx and UPS delivery services.

[Want to see how Amazon is moving into package delivery overseas? Read Amazon Moves Suggest Package Delivery Is on Its Mind.]

It also has its own distribution centers and same-day truck delivery systems so that it can deliver fresh vegetables to your door.

Amazon is now in the business of creating studio entertainment content to compete with HBO and Netflix. It is trying to find just the right device that captures consumers' loyalty and ties them more closely to Amazon, such as the Kindle Fire and Echo.

In effect, it's inventing new services as it goes along to load on top of those it's already got running. It's a big company, innovating like a small one, and daring Wal-Mart, Microsoft, and anybody else to keep up with it.

The one thing it hasn't done is report big profits. But when there's no one else who can exploit the new cloud services as effectively as it can, Amazon doesn't need to spend all day worrying about profitability. It keeps its universe expanding, and one day, with all those efficient services, the profits are sure to follow. Meanwhile, it's inventing what the new economy will look like, celebrating its tenth anniversary, and spending little time looking back.

Charles Babcock is an editor-at-large for InformationWeek and author of Management Strategies for the Cloud Revolution, a McGraw-Hill book. He is the former editor-in-chief of Digital News, former software editor of Computerworld and former technology editor of Interactive ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/18/2016 | 5:58:41 PM
Competitors have another view of Amazon's 10th
As Amazon cites 10th anniversary of AWS, Google cites its capture of Apple business on Google Cloud Platform versus Amazon. FedEx declares there's no way Amazon can compete with it (despite its leasing of 20 Boeing aircraft for package movement). And European regulators say Amazon may be discriminating against buyers who, in its view, return too many purchases. Everybody is objecting to the way Amazon does business? Well, Amazon has put a lot of people on the defensive with its expansionistic approach to its business.
Google in the Enterprise Survey
Google in the Enterprise Survey
There's no doubt Google has made headway into businesses: Just 28 percent discourage or ban use of its productivity ­products, and 69 percent cite Google Apps' good or excellent ­mobility. But progress could still stall: 59 percent of nonusers ­distrust the security of Google's cloud. Its data privacy is an open question, and 37 percent worry about integration.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.