Amazon CTO Vogels: Going Beyond Monolithic Virtualization

In a behind the scenes-style keynote at AWS re:Invent, CTO Werner Vogels shared some of Amazon’s strategy that informed the approach AWS took with virtualization. He also brought out clients such as The Vanguard Group to discuss their cloud migration journey and the results of those efforts.

There is no questioning the role virtualization plays in modernizing infrastructure; however, it has seen its share of permutations at Amazon according to Vogels. “Virtualization has been the bread and butter of the compute parts of any cloud environment from day one,” he said. The technical underpinnings of virtualization allowed cloud to grow into what it is today, Vogels said, with new boundaries being pushed in the process.

He described a phenomenon called root I/O virtualization tax where guest operating systems compete for the same I/O resources and trigger some undesirable effects. As virtualization scaled up, most guest OSs see significant jitter, Vogels said, and variations in latency on their network as they fight for the same network device. “We started to think about how we could radically change this and rethink virtualization to create a base of innovation for our customers,” he said. The issue could not be ignored because even newer architectures could be hampered by old style virtualization. Something had to change. “With traditional virtualization, there was significant overhead,” Vogels said.

To address such issues, he said Amazon built modular systems that applied lessons learned from software to revamp virtualization. “Basically, you should see the traditional virtualization world as a monolith,” Vogels said as he presented his case for change.

“What if we take the lessons from microservices, where you have small building blocks, where you can quickly innovate and apply those to the hardware world?” he asked. “Maybe we can change the world of virtualization.” This new frontier of virtualization includes tagged devices that have an API, he said. “It may be a hardware API like a PCI bus but still an API to virtual hardware.”

Vogels detailed some of the technical steps taken in the evolution of the AWS Nitro System, the bedrock platform for the Amazon Elastic Compute Cloud, which started from traditional virtualization then advanced into new territory. He said the first problem to be addressed was that of the network. “Just transferring an 8-gigabit file from [Amazon] S3 would result in literally hundreds of thousands of kernel traps,” Vogels said.

In order to solve that problem, he said they moved the network component to a separate card. The company learned a lot from offloading the I/O to a separate card, he said, then took another two years to become familiar with what it would take to offload processing to separate card that also sits on the same server.

Other steps in the process of evolving included moving EBS (elastic block storage) processing to a separate card, Vogels said. “No longer is volume processing in the network happening on the main CPU,” he said. Additional measures to change up virtualization led to benefits Vogels said included eliminating the typical jitter seen when interacting with an EBS device. This also allowed for the doubling of I/O operations per second to EBS and increased throughput. “After we offloaded it on to Nitro, basically all jitter has disappeared,” he said.

Asset management firm Vanguard shared its own story at re:Invent of moving the cloud, with AWS, transformed its operations. Jeff Dowds, information technology executive with Vanguard, said his firm has some 30 million investors, totaling $5.7 trillion in assets, making such moves no simple task. “We have no physical branches; we’re a digital firm,” he said. “From an IT perspective, we’re big and we’re complicated.”

Vanguard was running global data centers, mainframes, thousands of servers, lots of storage, thousands of apps, with 50,000 endpoints, Dowds said. “In our business, downtime is not tolerable.” Six years ago, Vanguard began its transformation rather than risk the perils of becoming a laggard. “We knew if Vanguard was going to stay competitive in the digital age, we needed to be better at the business of IT,” he said.

The organization wanted to accelerate the pace of innovation and deliver business value at startup speed, Dowds said. That plan included continuous integration/continuous delivery, DevOps, and microservices, he said, and cloud was the cornerstone to moving fast. Given the regulatory demands of asset management, Dowds said Vanguard started on a private cloud path because of concerns about public cloud security. Once those concerns abated, Vanguard pivoted to public cloud and chose AWS as cloud provider.

Here is how Vanguard got started on cloud migration and what they use:

“We had a traditional tech stack, heavily virtualized,” Dowds said. “We had big data platforms, monolithic applications. We had monoliths that were 30, 40, 50 million lines of code.” After a security apparatus with more than 150 security controls was in place, Vanguard felt comfortable migrating workloads to the cloud. Dowds said they established secure internet connectivity with Amazon Route 53 for DNS (domain name system), AWS Web Application Firewall, and Amazon CloudFront for CDN (content delivery network). Vanguard then migrated from VPN access to direct connect, which he said offered improved resiliency and bandwidth between facilities.

These steps allowed Vanguard to shut down its on-prem big data platforms. Dowds said they heavily use Amazon S3 (Simple Storage Service) and Elastic MapReduce, leveraging other machine learning capabilities while trying to get data closer to microservices.

Dowds said cloud transformation at Vanguard led to compute costs reduced at least 30%, building software 30% faster, and its deployment capabilities became 20X faster, improving the firm’s resiliency and ability to innovate. “Our end state is just about 100%, cloud-native architecture,” he said.