Software // Enterprise Applications
News
11/27/2007
11:19 PM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

PayPal Says Linux Grid Can Replace Mainframes

PayPal is currently processing $1,571 worth of transactions per second in 17 different currencies on about 4,000 servers running Red Hat Linux.

A Linux grid is the power behind the payment system at PayPal, and it's converted a mainframe believer. Scott Thompson, the former executive VP of technology solutions at Inovant, ran the Visa subsidiary responsible for executing Visa credit card transactions worldwide. The VisaNet system was strictly based on IBM mainframes.

In February 2005, Thompson became chief technology officer at the eBay payments company, PayPal, where he confronted a young Internet organization building its entire transaction processing infrastructure on open source Linux and low-cost servers. Hmmmm, he thought at the time.

"I came from Visa, where I had responsibility for VisaNet. It was a fabulous processing system, very big and very global. I was intrigued by PayPal. How would you use Linux for processing payments and never be wrong, never lose messages, never fall behind the pace of transactions," he recalled in an interview.

He now supervises the PayPal electronic payment processing system, which is smaller than VisaNet in volume and total dollar value of transactions. But it's growing fast. It is currently processing $1,571 worth of transactions per second in 17 different currencies. In 2006, the online payments firm, which started out over a bakery in Palo Alto, processed a total of $37.6 billion in transactions. It's headed toward $50 billion this year.

Now located in San Jose, PayPal grants its consumer members options in payment methods: credit cards, debit cards, or directly from a bank account. It has 165 million account holders worldwide, and has recently added such business as Northwest Airlines, Southwest Airlines, U.S. Airways, and Overstock.com, which now permit PayPal payments on their Web sites.

Thompson supervises a payment system that operates on about 4,000 servers running Red Hat Linux in the same manner that eBay and Google conduct their business on top of a grid of Linux servers. "I have been pleasantly surprised at how much we've been able to do with this approach. It operates like a mainframe," he said.

As PayPal grows it's much easier to grow the grid with Intel-based servers than it would be to upgrade a mainframe, he said. In a mainframe environment, the cost to increase capacity a planned 15% or 20% "is enormous. It could be in the tens of millions to do a step increase. In [PayPal's] world, we add hundreds of servers in the course of a couple of nights and the cost is in the thousands, not millions," he said.

PayPal takes Red Hat Enterprise Linux and strips out all features unnecessary to its business, then adds proprietary extensions around security. Another virtue of the grid is that PayPal's 800 engineers can all get a copy of that customized system on their development desktops, run tests on their raw software as they work, and develop to PayPal's needs faster because they're working in the target environment. That's harder to do when the core of the data center consists of large Unix symmetrical multiprocessing boxes or mainframes. In neither case is it cheap to install duplicates for developers, he said.

PayPal "pays very close attention to the Linux kernel development process" lead by Linus Torvalds and the kernel maintainers because future capabilities are being debated and resolved through the process, he said.

PayPal has experimented with virtualization and is watching carefully developments in open source virtualization, still a young field. "One place we see the kernel process at work is in virtualization," Thompson said. VMware's ESX Server can run Linux, as can the open source Xen hypervisor; both work outside the Linux kernel but can be linked to its internal operations. A year ago, Torvalds approved the addition of a contributed Kernel Virtual Machine, which runs inside the kernel and makes use of the kernel's own memory management and other functions.

"If we could fully virtualize our middle tier, that would be another step of cost advantage," said Thompson. More fully virtualized data centers also would allow him to shift workloads across the grid, depending on time of day and traffic volumes, which would lead to additional savings.

"We'd love to shift processing capacity to workloads. That would be a tremendous benefit," and slow the need to buy and manage blade servers, even as PayPal continues to grow, he said.

Virtualization also would help him avoid building out another "power center" or new unit of the data center located close to cheap power. Avoiding added power costs is a much higher priority than it was a few years ago, he said.

But he's not ready to virtualize batches of servers here and there. He wants a plan that will allow "virtualization with a layer of intelligence on top of it" to manage virtual resources. He is experimenting with VMware, but before he implements virtualization throughout the data center, he'll wait until he sees the right intelligence materialize to make it manageable.

Does relying on Linux worry him when Acacia Research, through its subsidiary IP Innovation, filed suit in October against Red Hat and Novell for violating its patent portfolio? And Microsoft claims its patents are violated by Linux? Thompson said: "I'm not worried about those statements. I'm familiar with the issue but I'm not worried about it. We have people in our business who are on top of it."

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
jeffmsullivan
50%
50%
jeffmsullivan,
User Rank: Apprentice
9/19/2013 | 10:52:42 PM
re: PayPal Says Linux Grid Can Replace Mainframes
Another case of getting cost of acquisition mixed up with cost of ownership. People costs: 800 engineers making an average of $60,000/year (conservatively) would be $48 million per year. A mainframe infrastructure of 3 z12 at an entry level of $50 million can be maintained by about 10 people or about $1 million per year. Machine costs: 4,000 servers at $2,000/server is $8 million. It is not the machine cost that can only be figured in but the ownership. I maintain that the reason this was kept on Linux grid is one of...costs. This system would be in the billions to report. By the way, z12 machines can run Linux, too and without so so so many servers. (and no need for a failover server either!
Building A Mobile Business Mindset
Building A Mobile Business Mindset
Among 688 respondents, 46% have deployed mobile apps, with an additional 24% planning to in the next year. Soon all apps will look like mobile apps and it's past time for those with no plans to get cracking.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - July 22, 2014
Sophisticated attacks demand real-time risk management and continuous monitoring. Here's how federal agencies are meeting that challenge.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.