Facebook Data Center Showcases Open Compute Work

H-1B Visa Program: 13 Notable Statistics

The Open Compute Project has taken strides forward with its Networking Project, including a data center network fabric, installed by Facebook at its newest data center, which will be contributed to the project.

The Altoona, Iowa, data center opened Nov. 14 and is the first Facebook facility to include a 40-Gbps Ethernet network fabric that links every pod, or set of 48 server racks, to every other pod in the data center. The fabric design gets away from the constraint of a server cluster being limited by the lack of bandwidth connecting it to other clusters.

"We challenged ourselves to make the entire data center building one high-performance network, instead of a hierarchically oversubscribed system of clusters," wrote Alexey Andreyev, Facebook network architect, in a blog post on the fabric.

[Take a look inside another Facebook data center. See Facebook's Futuristic Data Center: Inside Tour.]

That means Facebook has found a way to "disaggregate" what had been a few high-performance and vendor proprietary switches managing each cluster's traffic into smaller units spread throughout the data center. "Instead of the large devices and clusters, we broke the network up into small identical units -- server pods -- and created uniform high-performance connectivity between all pods."

And that means Facebook can extend high-performance networking throughout its data centers whenever it chooses. Its more modular networking approach is scalable beyond the limits of its previous cluster management switches.

In addition, Facebook has been enhancing its Wedge top-of-rack switch, whose specifications are part of the Networking Project, noted Najam Ahmad, VP of network engineering at Facebook, in an interview.

"We didn't have a top-of-rack design laid out initially. We spun up the Networking Project (a year ago) without having a design ready to go," he recalled.

Facebook, through the Open Compute Project, is trying to make basic components of data center hardware available through open source designs. "The primary job was to disaggregate the network appliance, most often a proprietary chip, operating system, and hardware built by one vendor," he said.

Among other things, Open Compute advocates are seeking to add a micro-server to a standard switch or other network hardware device through which it may be programmed and managed on a dynamic networking basis. That's a hardware idea that meshes with a software-defined network, with a network controller reaching out to such devices with sets of instructions.

Mellanox and Dell have produced Wedge-specified switches, he noted.

At the end of October, the Open Compute Project announced that it has achieved the Open Network Install Environment (ONIE), an open source and freely available way to boot up network switches, after testing them. There are already ways to boot switches, but they are either proprietary and billed with the purchase of products, or they are a published standard that's still accompanied by a fee.

ONIE, contributed by Cumulus Networks, "is one of the fundamental pieces that we've done and announced," said Ahmad.

Cumulus Networks produced ONIE as an open source boot loader for the switch. It does initial testing of the hardware, then can boot it up for operations with the software assigned to it. "It's like the BIOS on a compute server," noted Ahmad.

Our latest survey shows growing demand, fixed budgets, and good reason why resellers and vendors must fight to remain relevant. One thing's for sure: The data center is poised for a wild ride, and no one wants to be left behind. Get the Research: 2014 State Of The Data Center report today. (Free registration required.)