AWS Helps Fuel Lyft's Expansion - InformationWeek
IoT
IoT
Cloud // Infrastructure as a Service
News
8/22/2016
10:05 AM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

AWS Helps Fuel Lyft's Expansion

Lyft, the San Francisco ride-sharing company and Uber competitor, is thriving on AWS IT services and looking to ring up its first $1 billion. Here's what IT pros can learn from the example.

9 Cloud Skills IT Pros Need Now
9 Cloud Skills IT Pros Need Now
(Click image for larger view and slideshow.)

As a startup company, Lyft wasn't so different from many other young companies. It skipped the data center building phase, never bought servers, and instead established its business in the cloud.

"It was 100% based on Amazon Web Services. We had three servers in US East (Ashburn, Va.), each in a different availability zone," recalled Chris Lambert, the CTO of Lyft, during a recent talk during the Amazon Summit in New York on Aug. 11.

That decision was made in 2012, when the company got started. Four years later, "We have thousands of servers today, powering over 150 different microservices," said Lambert in an email exchange with InformationWeek. Lyft is on a rate of $1 billion in revenues. But, says Lambert, it still doesn't have a data center.

"We're still 100% on Amazon but we're taking advantage of many more Amazon technologies than just EC2," Lambert said, after being invited to the stage during a keynote by AWS CTO Werner Vogels.

Lyft is an example of how a company can start out small with Amazon and within a few years find it needs some of the business IT services that Amazon has evolved into providing.

(Image: Jason Doiy/iStockphoto)

(Image: Jason Doiy/iStockphoto)

For example, Lyft uses Autoscaling, DynamoDB, Load Balancing, and Kinesis data streaming.

In addition, the company started out using Amazon's DynamoDB service for the data it gathered on each ride, including the GPS data, after outgrowing its own server-hosted database. The data built up quickly, but Lyft managers liked the ease with which DynamoDB was able to scale with the growth.

"It was so simple to scale out. We had two knobs. One was for reads and one was for writes," Lambert recalled.

The engineering team didn't have to worry about "chunk migration" or look for an alert if when things went awry. "It just works," Lambert said at the summit.

In an additional email comment, Lambert wrote:

We now have hundreds of different DynamoDB tables in production, across many different production services. We're not releasing any numbers on storage size, but it's fair to say that we haven't been constrained by any capacity limits on DynamoDB.

In part, keeping the focus on the rides and not the compute infrastructure is a constant goal. Lambert recalls Lyft's first application on EC2 was so simple that the two engineers who conceived it could see data indicating the first passenger had arrived at his destination, but they weren't sure they should believe it.

"They said, 'Maybe it worked ... we actually don't know,'" he told the summit audience. So they got the driver's phone number and called him.

"'Did you just drop off Darrell?' He said, 'yes.' I don't know who was more surprised: The engineers that it worked or the driver that we had called him," Lambert recounted. The driver probably figured that would be his experience with every drop-off, he joked.

Expansion in Real-Time

That first passenger drop-off occurred May 31, 2012. By March 2014, Lyft was active in San Francisco, Los Angeles, and a few other cities. However, it decided it was time to make a big move.

Again, it wanted to maintain simplicity of operations but at a much larger scale.

"We decided to expand into 24 cities. In engineering, we said, 'Let's get out of the way. Our infrastructure should scale up seamlessly,'" Lambert said. At that point, the engineers invoked Amazon's Auto Scaling service.

Auto Scaling let Lyft's operation expand to cope with its busiest period of the week -- Saturday night -- and it helped Lyft in another way.

"We do eight times the rides at our peak time on Saturday night compared to Sunday morning," Lambert noted. Auto Scaling allowed Lyft operations to automatically shut down servers that it no longer needed. "Scaling down is equally important. You save money when you don't use all this capacity that you don't need."

Lambert added to his summit comments in an email message:

From day one, Lyft has always had a solid track record with service orchestration and configuration management. This got us a lot of the way towards being prepared for autoscaling, but there were some subtleties around log rotation and draining event queues that we wanted to button up, and we were able to do that before the 24-city launch in March. 

Once we made the move to autoscaling, scaling our service tier was no longer an operational task that our engineering teams had to think about, freeing us up to invest more in other areas of the business.

Lyft continued to expand its use of Amazon IT services over the next year, building up to August 2014, when it had its "single biggest launch, Lyft Lines, our shared ride service," Lambert said.

Lyft had been collecting reams of data on its rides, including all the GPS points recorded, and turned that data over to its data scientists. Every few months they ran simulations on it in Amazon Redshift, asking, "What might be possible in the way of new transportation modes," Lambert said.

"We found often during peak times, if two users request rides within a few minutes of each other over similar routes, we can offer them the option of sharing the ride, with up to a 50% savings possible," Lambert said. "We realized at peak times, 90% of our rides were co-similar enough that we could actually build a shared rides product... "

Getting More From Data

The data mining in Redshift and resulting application proved extremely valuable to the young firm. It helped differentiate it from competitors, such as Uber. "It's our biggest driver of growth in our biggest markets, and is a feature of the company," said Lambert.

"We have a number of teams that are working on data driven approaches to improve the Lyft Line experience, even today," Lambert added in an email message.

To some extent, Lyft appears to have become habituated to the services that Amazon was offering and willing to experiment with ways to make them useful to its business. One such service, Kinesis, is a streaming service that many people tend to think of as something you would use to collect data from devices on the internet of things.

Lyft engineers realized it might fit something else they wanted to do.

By August 2015, a year after the Lyft Lines application started running, Lyft was experiencing a build-out of small services designed to operate independently in the cloud. By August 2016, it had 100 microservices in operation.

"We had a proliferation of microservices, all serving production traffic, and we wanted an easy, simple way for them to communicate with each other. We built this pub/sub system on top of Kinesis that funnels every single production event through this system," he said.

"Every time you open the app, every time you request a ride, every time the car actually moves, it goes through this system," he told the crowd.

[Want to see what AWS customers are using? Read AWS S3, Data Transfer Among Its Most Popular Services: Report.]

As software events stream through Kinesis, other applications can tell which events they wish to be notified of.

"Any other service at Lyft can subscribe to it," Lambert said. "I care about credit card adds (new credit cards added to personal accounts). So when a customer adds a card, our fraud system can say: 'Every time a user adds a new payment system or removes a payment system, let me know because I want to rescore that user's fraud risk.'"

Kinesis event and data streams will allow Lyft to constantly check its pricing versus passenger traffic, implementing demand-based pricing during peaks. That's something that Lyft doesn’t want to get wrong and yield advantage to competitors, he said.

Lyft's systems appear to be doing the job. As the company creeps up on $1 billion in revenue, it's carrying 10,000 people at any given moment and delivering 14 million rides a month. "Downtime is not an option for us," said Lambert. So far, AWS has been "reliable and available."

What he likes most is the way Lyft concentrates on delivering rides and the customer experience of these rides, while Amazon concentrates on the infrastructure. "This is a complex infrastructure with a lot of moving parts -- but it's simple for us to maintain," he said.

Charles Babcock is an editor-at-large for InformationWeek and author of Management Strategies for the Cloud Revolution, a McGraw-Hill book. He is the former editor-in-chief of Digital News, former software editor of Computerworld and former technology editor of Interactive ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
9/2/2016 | 1:34:52 PM
Medallions outmoded, but taxis have to buy them; ride-sharing firms don't
Ride-sharing has become so dominant in some cities that locations such as the State of Massachusetts and the City of Austin, Texas, are taxing Lyft and Uber rides to subsidize the struggline taxi industry. Critics say that forces innovators to subsidize losers, but in my view, the taxi industry is forced to pay municipalities huge sums for the medallions that give them the right to operate a service. Lyft and Uber have no such tax on their operations. In Boston, the value of a medallion has dropped from $700,000 to less than $200,000, so the state  imposes a 20-cent tax on each ride, with 5 cents going to the taxi companies. It may not be a great idea. I'm not sure it levels the playing field. But it addresses how part of innovator's success can be based on avoiding taxes. Amazon.com for years avoided state sales taxes that book stores paid. And we know what happened to the bookstores. 
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
8/24/2016 | 3:20:25 PM
Uber and Lyft: culture clash?
jastroff, I think the company cultures are quite different, even though they appear to be involved in the same ride-sharing type of business. Uber is more brash. Lyft has been less prone to clash with local authorities. Uber has been more aggressive in opening its systems to third parties. It brought out a public-facing API in 2014. Lyft did so in March 2016. And if Uber has lost $1.2 billion so far this year, as reported, then it's probably a more aggressive investor/spender in the market than Lyft.
jastroff
50%
50%
jastroff,
User Rank: Ninja
8/24/2016 | 10:40:02 AM
Re: Lyft Lines helped in Uber competition
Why do you think Uber didn't buy Lyft recently? Price? 
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
8/23/2016 | 5:17:50 PM
Lyft Lines helped in Uber competition
A family member who drives for Lyft told me a recent ride consisted of an Uber manager sent to recruit him as a driver. The competition put upon Lyft by Uber has been intense, and Lyft has responded with Lyft Lines and other innovative measures.
SaneIT
50%
50%
SaneIT,
User Rank: Ninja
8/23/2016 | 8:24:04 AM
Enabling start ups
"It was so simple to scale out. We had two knobs. One was for reads and one was for writes," Lambert recalled.

 

I love this and feel that this is one of the best reasons for a start up to partner with a service provider like AWS early on.  If they had started out on local servers and had to migrate, nothing would have gone as smoothly.  You may give up a little bit of framework freedom but the framework isn't lacking at all.  More and more developers are building inside AWS from the start now so I can only see this becoming more common.  The flexibility can make a small company feel very big in a situation similar to Lyft and their rapid expansion. 
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends for 2018
As we enter a new year of technology planning, find out about the hot technologies organizations are using to advance their businesses and where the experts say IT is heading.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll