Software // Information Management
08:31 AM
Connect Directly

Facebook Open Compute Project Shapes Big Data Hardware

Big data practitioners like Facebook, Goldman Sachs and Fidelity are setting the standards for cheaper, more efficient servers and systems from the likes of Applied Micro, AMD, Dell and Intel.

Big Data's Surprising Uses: From Lady Gaga To CIA
Big Data's Surprising Uses: From Lady Gaga To CIA
(click image for larger view and for slideshow)
Facebook and the Open Compute Project (OCP) announced Wednesday that they've made huge strides toward the goal of setting standards for the most efficient server, storage and data center hardware available for scalable computing.

Facebook launched OCP 18 months ago hoping to crowdsource the problem of creating better hardware for high-scale computing. From its start with one member, Facebook, and 200 participants, the group now has more than 50 member companies and saw more than 2,000 participants attend this week's Open Compute Platform Summit in Santa Clara, Calif.

OCP says its standards promise to deliver hardware that is 24% more energy efficient and 38% more cost efficient, on average, than so-called commodity hardware. The group is working on specs for storage, motherboard and server design, racks, interoperability, hardware management, and data center design.

"We know what we need and we see the current and future challenges that the deluge of data, analytics, compute power and these massively scaled data centers are bringing forward," said Frank Frankovsky, VP of hardware design and supply chain at Facebook and chairman of OCP's Open Compute Foundation, which is modeled after the Apache Software Foundation.

[ Want more on Facebook big data projects? Read Meet Facebook's Graph Search Tool. ]

Hardware vendors have also joined OCP and are designing hardware to its specs. AMD and Intel announced in May collaborative work with OCP members Fidelity and Goldman Sachs to develop new boards to suit their financial processing workloads, and both chip makers presented finished products at this week's event.

Intel has contributed specs to OCP for Silicon Photonic interconnect technologies that already surpass 100 gigabits per second -- nearly twice the speed of the fastest interconnect technologies currently available. Presenting on the technology, Intel CTO Justin Rattner said the products will soon be commercially available from Intel and would solve data I/O constraints both inside servers and in connecting servers within racks and racks within data centers.

"It has been a decade-long effort to create the technology, and it has enormous potential due to its combination of speed, low power consumption and low cost," Rattner said.

Dell also presented at the event, showing off Intel- and ARM-based OCP-compliant servers that will soon be commercially available. Other manufacturers working on or delivering OCP-compliant hardware and components include Applied Micro, Calxeda, Delta, Emerson and Hyve Solutions.

Many of the cost savings in OCP-standard products come down to two design themes: commonality and disaggregation. In an example of commonality, OCP unveiled a "common slot architecture" that will enable ARM and X86 chips to coexist on the same motherboard. The design will give end-user organizations flexibility to test and configure servers for best possible performance and then reconfigure as workloads change without facing proprietary design constraints. The spec is being supported by AMD, Applied Micro, Calxeda and Intel.

Disaggregation is about separating and modularizing storage, compute, interconnects, power, cooling and other components so companies can custom configure to their workload requirements.

This approach also supports smarter technology refreshes, so companies can swap out and replace quickly evolving components, such as CPUs, while keeping in service slowly evolving components, such as memory and network interface cards.

"You might be five generations behind on the processor while 80% of the other components of the server are still good," Frankovsky said, noting that monolithic designs don't let you replace dated components such as CPUs. "Smarter technology refreshes not only make sense from a financial perspective; think about how much less IT equipment is going to go into the waste stream."

While many OCP members are hyper-scale Internet companies, the point was made Wednesday that the group's efforts will benefit companies with mainstream computing requirements within a few short years.

"Hadoop, Hive, Hbase and other new platforms are accelerating the growth of big data adoption, so our big data challenges of today are everyone's big data challenges of tomorrow," said Jay Parikh, Facebook's VP of infrastructure engineering.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
D. Henschen
D. Henschen,
User Rank: Author
1/17/2013 | 11:18:11 PM
re: Facebook Open Compute Project Shapes Big Data Hardware
Silicon photonics sounds like it has huge advantages over copper-wire-based interconnects. Components that used to have to be on the motherboard (close to the processing power) can now be "disaggregated" (meaning separated from the board). The photonic components themselves can be stamped mass-production-style, just like chips, for low cost. The cables can be much longer than copper without speed penalties, thus they're suitable for connecting servers within racks and racks across data centers as well as components inside the server.
D. Henschen
D. Henschen,
User Rank: Author
1/17/2013 | 6:17:49 PM
re: Facebook Open Compute Project Shapes Big Data Hardware
Silicon photonics sounds like it has huge advantages over copper wire. Components that used to have to be on the mother board can now be "disaggregated." The components can be stamped, just like chips for low cost. The cables can run longer than copper without speed penalties, thus they're suitable for connecting racks across data centers.
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on for the week of October 9, 2016. We'll be talking with the editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll