How To Build A Modern Data Center

Borrowing ideas from Google, Facebook, and Microsoft, Vantage is building three wholesale data centers in Santa Clara that focus on energy efficiency.
Analytics Slideshow: 2010 Data Center Operational Trends Report
(click for larger image and for full slideshow)
A site's power distribution features are not something that just any company can go back and revise to save energy. They have to be built into the original design. Vantage's Walsh Avenue campus is supplied with electricity through an on-site, 50-megawatt substation with two transformers. The substation sits on a "self-healing" transmission ring with other substations located on it. The Vantage substation could malfunction and the other substations would still ensure that the site got power it needed, he said.

Most enterprise data centers lack a 50-megawatt substation nearby. In one case, an Internet entrepreneur related how his servers went down at his co-location facility because the site was strapped for power and couldn't get more. All seemed fine until a co-lo employee to the nearby lunchroom one day, pushed down the toaster button, and the data center went dark.

Trout said that's why most data centers never tap more than 80% of the power being made available to them. They want a buffer, a margin of error, in case of an unanticipated toaster event. At the Vantage site, however, the "robust design" calls for the substation to deliver the power needed without a buffer. Trout said the data center is "using 1.0 of the electricity it brings in the building, instead of .8."

In addition, the power is stepped down from transmission line voltage at the substation to 480 volts as it's carried to different sections of the data center, instead of the 220 or 110 that would be more typical. Facebook did the same thing, Trout said, delivering 480 volts close to the servers because the technique results in less transmission line power loss. Under normal circumstances, the delivery of the power consumes a percentage of the resource due to resistance in the line. The higher the voltage, however, the less the loss. Voltage isn't literally a measure of an amount of electricity so much as an indication of the pressure behind it.

Once it's close to the server racks, it's stepped down to 12 volts and passes through a Vantage innovation, an insulated gate bipolar transistor (IGBT) unit. It's unusual to use a IGBT at this stage because it's actually a specialized semi-conductor that can allow alternating current to pass through it. If the AC power disappears, it allows a direct current supplied by batteries to instantly replace the loss.

The IGBT device acts as the key component of the Vantage data center's uninterruptible power supply. It both conveys normal AC current and can convert direct battery current, DC, into instant replacement alternating current. It has to perform this function in case of a power outage for only for 5-8 minutes, the amount of time the batteries can sustain the data center load, but that's enough to get the backup generators fired up and running, Trout said.

Neither Facebook nor Google use the IGBT approach, to the best of his knowledge. Each data center use some form of uninterruptible power supply and typically transmits 88% to 90% of the electricity coming into the data center through it. It's necessary for the UPS to sip power from the normal supply to insure that its batteries are fully charged. The Vantage IGBT approach gets 96% of the power coming through the UPS, he said.

Although Vantage is trying to use all the power coming onto its premises, it has through its design the ability to get more power from the substation. Trout called it "2n redundancies all the way to the server" or twice the amount of power needed can be delivered by the system. If one transformer in the substation dies, for example, the surviving transformer can still deliver all the power needed.

Likewise, if smaller transformers die in the step-down process, other transformers kick in to keep the servers running. It's a deliver just-in-time strategy, instead of keep (and waste) a power reserve.

The building of this distribution into Vantage's first whole sale data center on its campus, completed in mid-February, will be used again in the second phase now underway and a third still to come. Trout said the big step energy savings are occurring now, with examples like and Microsoft, as well as Google and Facebook, paving the way.

There will be more to come, but the art of data center design has finally gotten around to addressing its major sources of wasted power. "We may move one day from 1.07 to 1.06 or even 1.05," he said. But there will be no more announcements of 38% power savings in a new data center versus the one that came before it, he said. Today is the time for dramatic gains. Tomorrow's gains will come much harder.