PayPal Revamps Data Center For Agility

New mobile and online payment processes keep showing up in the marketplace, including high profile efforts from Apple and Google. With competition increasing, PayPal knew it needed a faster and more agile way to add new features and updates. And that meant some big changes in its development strategy and its data center.

A little over three years ago, "We were still following the traditional waterfall model," recalls CTO James Barrese. Barrese joined PayPal in 2011 and concluded the company "had gotten too conservative" in the process of growing big. To respond to competition, PayPal needed to add to its mix of services more frequently. Moving to Agile development was part of the needed change, but that wasn't enough unless the infrastructure could be just as nimble. "Launching a small service required more than 100 tickets and took three weeks to provision new servers," Barrese says.

So PayPal transitioned its data center infrastructure into a private cloud using OpenStack cloud management software. The result is that PayPal can execute product cycles "seven times faster than a year ago," Barrese says, and its development platform "lends itself to building fast and highly scalable network applications." When those apps are based on the Node.js environment, they use 40% fewer lines of code than required in previous applications and require 20% of the time to develop.

"Our culture was transformed by some of these initiatives," Barrese says. It's not just agile development, shorter product cycles, and frequent product updates. The business itself needed to become more agile and more quickly convert market needs into production software.

"We did it primarily for agility," says Sri Shivananda, VP for global platform and infrastructure, of the data center's private cloud transformation. "We wanted to prove to our internal application development folks that if they were asking for a few hundred machines, they were going to be available on demand." Now a development team's resources can be provisioned in a few hours as opposed to three weeks.

The move required both technical and skill changes. The IT staff had to learn to work with more open source code, as it introduced OpenStack's KVM default hypervisor and virtual machine management to replace certain VMware operations software. (VMware software remains in use for establishing the virtual network that the VMs use.) PayPal is now a 100% OpenStack private cloud in its stateless, customer-facing Web and mobile applications. And the OpenStack private cloud also runs certain mid-tier services supporting business logic, says Shivananda.

Agility Brings A Talent Dividend

Instead of losing some of its best technologists, as had been the case when it was using waterfall procedures, PayPal is attracting more premium talent as an agile and cloud-based organization. In February 2014, it brought in 
Danese Cooper, former CTO of Wikimedia Foundation, home to Wikipedia, to re-architect PayPal's back-end systems to make better use of open source. PayPal also brought in Arnold Goldberg, its VP of customer engineering, from Box, and Bill Scott, VP of payment products, from Netflix. Its new orientation toward Node.js for interactive applications helped land Doug Crockford, an originator of JavaScript and now senior architect at PayPal.

PayPal currently has 162 million active customers, using 26 currencies in 203 markets around the world. It transacts $228 billion in payments a year. With its converted infrastructure and culture, PayPal will be able "to evolve and grow our infrastructure and back-end technology" in these markets, instead of lagging them, says Barrese.

"We want to empower our talent to be more strategic, innovative, and to focus on longer-term initiatives," he says.