Rise of Data and Asynchronization Hyped Up at AWS re:Invent

The leadership at Amazon.com and AWS seem to see world awash in data that can make the most from cloud-based, asynchronous architecture to keep up with escalating demands.

Last week’s AWS re:Invent conference in Las Vegas, held in-person and online, saw AWS CEO Adam Selipsky give a keynote that highlighted the vast mountains of data emerging on the digital landscape while also framing his company, naturally, as a resource that a broad breadth of clientele turn to in such an environment.

For example, Selipsky cited the migration of PrivatBank in Ukraine, which he described as that country’s largest bank serving 40% of the populace, to AWS while facing an ongoing military invasion. “Under the specter of war, they’ve managed to move all of their operations to the cloud -- 330 IT systems and four petabytes of customer data to AWS in just 45 days,” he said.

Big Data, More Cloud

The pace of data growth is only expected to further ramp up, Selipsky said, making the flexibility of the cloud a potentially attractive option for organizations that need to scale with that growth. “Analysts predict that within the next five years, we’re going to create more than double the amount of data produced since the dawn of the digital age,” he said. “Managing the scale and the growth of that data is a huge challenge and opportunity for every organization.”

Selipsky offered a glimpse of the a few AWS customers who face hefty data demands. Expedia Group, for instance, processes more than 600 billion predictions per year, powered by 70 petabytes of data, he said. Samsung’s 1.1 billion users, he said, make 80,000 requests per second. Selipsky said Pinterest stores one exabyte -- one million terabytes -- on Amazon S3 Simple Storage Service). “Data is at the center of applications, of processes of business decisions and is the cornerstone of almost every organization’s digital transformation,” he said.

As Selipsky talked up the necessity of analytics tools for all data types and use cases, he brought on stage Biljana Kaitović, executive vice president of digital and IT with utility company Engie, an AWS client. Kaitović said her company wants to support greater energy efficiency, including low-carbon energy efforts and services. “To optimize and forecast production levels of our renewables, we use a lot of data and advanced analytics,” Kaitović said. “For predicting equipment failure before it occurs, we use predictive analytics, machine learning, and the Internet of Things.”

She said decentralization in the energy sector has led to an increase in small, local energy producers and consumers within microgrids. This includes a rise in solar-powered homes and electric cars, which are shifting some of the dynamics in the utilities space. “To balance supply and demand in this very complex system, we need to use a real-time energy management system,” Kaitović said.

A World of Asynchronous Compute Architecture

In his keynote held later in the week, Werner Vogels, Amazon.com’s vice president and CTO, said the world is asynchronous and might actually be better for it. “There is absolutely nothing that is synchronous in this world,” he said. “And if it is, we really wouldn’t like it.” This includes building computer systems, Vogels said, and developing the overall digital world. The options and variations that asynchronous computer architecture offer can mean there are possible ways to move forward even if there is a digital catastrophe. “When I think about asynchrony,” he said, “it is that we should make progress under all circumstances, no matter what happens.”

The concept of asynchrony has already been at play at Amazon, Vogels said, with the development of the S3 offering. “We wanted to make sure that system would make progress under all circumstances,” he said, “no matter what’s involved; no matter what is failing.” However, he also said synchrony in compute can seem easier, at first glance, when it comes to latency and throughput. “Synchrony is a simplification. It’s just something that makes it easier for us to write our programs,” Vogels said, with a caveat. “Synchrony is an illusion. It’s something that we build over a world that is asynchronous.”

Because it was believed that asynchronous programming was difficult, he said, operating systems tended to have restrained interfaces. “If you wanted to write to the disk, you got blocked until the block was written,” Vogels said. Change began to emerge in the 1990s, he said, with operating systems designed from the ground up to expose asynchrony to the world. “Windows NT was probably the first one to have asynchronous communication or interaction with devices as a first principle in the kernel.” Linux, Vogels said, did not pick up asynchrony until the early 2000s.

Werner Vogels, CTO of Amazon

The benefit of asynchrony, he said, is it is natural compared with the illusion of synchrony. When compute systems are tightly coupled together, it could lead to widespread failure if something goes wrong, Vogels said. With asynchronous systems, everything is decoupled. “The most important thing is that this is an architecture that can evolve very easily without have to change any of the other components,” he said. “It is a natural way of isolating failures. If any of the components fails, the whole system continues to work.”

Vogels also said asynchronous architecture evolves rather than be built in one shot. “Decomposing into small, well-understood building blocks is the fundamental part of building these asynchronous, loosely coupled, event-driven systems,” he said.

As a retailer, Amazon started in a monolithic system, Vogels said, that eventually reached end of life and could not scale any more. “It was all driven by shared resources in the backend, a whole battery of databases that were direct accessed,” he said. When the company wanted to innovate further, it could not because it could not evolve its system at that time. “We moved over to service-oriented architecture,” Vogels said. “We learned from that; we went to microservices architecture and now Amazon retail is basically a very large collection of microservices running over a shared infrastructure.”

What to Read Next:

Should Amazon Spin Off AWS?

Cloud, Data, and Political Protests Mark the 2022 AWS Summit

AWS CTO Vogels on Cloud Eliminating Constraints on Innovation