Photobucket Pictures Its Future On OpenStack - InformationWeek
Cloud // Infrastructure as a Service
02:25 PM
Connect Directly

Photobucket Pictures Its Future On OpenStack

Cisco's version of an OpenStack private cloud helps Photobucket ease implementation and management of networking issues.

Photobucket is using a Cisco-built OpenStack cloud to better compete with the heavily financed Instagram, Pinterest, and Flickr as a lifetime destination for users' photos.

Photobucket stores personal photos, provides editing tools, and generates a link to users' photos so that they can be displayed in documents or on other websites without being uploaded to those destinations.

The 10-year-old, 74-employee company was once all the buzz. Now it has to compete with Pinterest, which wants to be everyone's lifetime scrapbook site for photos, videos, and documents. In late October, Pinterest garnered $225 million in venture capital to expand its services, following $200 million it received in February. Its estimated valuation is $3.8 billion.

Photobucket knows it lags behind Instagram, the instant sharing service that adds a telegram-like ability to deliver images wherever its users want. But Photobucket figures it has edged ahead of Flickr, because it hosts 4 billion transactions a day, has links to 3 million websites, hosts 20 million unique visitors a month, and stores 14 billion pictures.

 [Want more on how Photobucket uses mobile applications? See Help Wanted 2.0: SMB Recruiting Goes Mobile.]

Given the nature of the competition, Photobucket sought to operate in the most efficient manner possible. It knows that fractions of a penny saved in one part of its operations would support other operations as a long, competitive race runs itself out.

These challenges prompted Mike Clark, Photobucket's co-founder and CTO, and Jay Kistler, its director of operations, to look for the best way to reorganize its infrastructure in 2012. Kistler realized that server utilization was only in the 15%-20% range for 800 bare-metal servers. Virtualization would improve that, but how should it virtualize?

Key staff members began OpenStack pilots in what was still an early version of the open-source cloud software. The staff members "dove in" and found OpenStack a somewhat unruly amalgamation of code at the time, Kistler told us. "It's not for the faint of heart." Particularly baffling was the Quantum networking part of OpenStack (now renamed Neutron), which proposed a new way to do networking in the cloud. Kistler and Clark weren't too sure how its software-defined approach to virtualized networking was going to play out.

"We actually are a strong command-line company," Kistler said. IT operations were comfortable with Cisco's converged approach to racks of servers, network switches, and storage in its unified computing system (UCS) architecture, but they didn't want an all-proprietary Cisco cloud.

Cisco was happy to help Photobucket implement a Cisco version of OpenStack on top of UCS. Now, instead of 800 bare-metal servers, Photobucket operates MySQL databases serving content from 350-400 physical servers. Hadoop analytics (essential to understanding what users are doing and how they're using Photobucket servers) also runs unvirtualized on a bare-metal cluster. The rest of operations run on the OpenStack Cloud, which has cut an additional 350-400 servers down to two UCS clusters -- one running in Denver and one in Phoenix. Both are operated from one management console as a single OpenStack cloud.

The Cisco distribution of OpenStack includes a plugin for OpenStack's Quantum networking that works with the Nexus 1000 switch for multiple virtual machines and Cisco's accompanying networking fabric. Clark said choosing this option saved Photobucket the task of deciphering an emerging and rapidly changing part of the OpenStack cloud. "It requires some real technical acumen to stand up an OpenStack cloud."

Photobucket has gone its own way with the Salt open-source configuration and deployment engine, written in Python, the preferred language of the firm's developer teams.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Li Tan
Li Tan,
User Rank: Ninja
11/25/2013 | 2:34:00 AM
Re: KVM live migrations?
OpenStack is getting from its infancy stage to a rather mature stage - you can do the live migration to it by utilizing the tools available. Regarding KVM, I am not so experienced with it. My company have used ESX intensively in IaaS, though.
User Rank: Ninja
11/24/2013 | 5:35:21 PM
Re: KVM live migrations?
One might wonder when reading this article why Photobucket has waited so long to virtualize chronically underutilized infrastructure. 

But that's first blush. When you read through some of the methodology behind all of this, it is clear that today's advancement of OpenStack has progressed to the point where it fits this organization perfectly, despite the negative attention the solution has faced in the media. 
User Rank: Strategist
11/22/2013 | 3:40:34 PM
KVM live migrations?
Photobucket uses KVM's Live Copy on top of UCS servers and Netapp's FAS storage systems, a combination known as FlexPod. When doing an update, administrators migrate VMs away from the target host, then back again, "with no disruption of service... Once cooking, it's really easy," said Kistler. Anyone else use KVM for live migrations in production?  
Register for InformationWeek Newsletters
White Papers
Current Issue
Cybersecurity Strategies for the Digital Era
At its core, digital business relies on strong security practices. In addition, leveraging security intelligence and integrating security with operations and developer teams can help organizations push the boundaries of innovation.
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll