A tour of Microsoft's gargantuan, under-construction San Antonio data center reveals a state-of-the-art IT infrastructure on an immense scale.
There are two huge water towers on site for storing chilled water, one for each half of the data center. Each is so big -- roughly 30 feet in diameter by 30 feet tall -- that it could easily serve as a water tower for a small town. Microsoft's San Antonio data center receives a large supply of nuclear and wind energy as well as a large recycled water supply. After energy, Manos believes, water will be the next raw material to affect data center expense calculations.
The scale of the San Antonio facility is so large that Microsoft has had to develop software to manage it, since no independent vendors can easily do so. For example, the software can run a full simulation of the data center running on all cylinders and run tests for metrics and measure loads and other things. The company can also measure and calculate down to the smallest elements, such as how many watts the data center uses and how much revenue is gained per search query, for example.
Manos has a group called Research and Engineering that's operates like data center design R&D arm, with 40 to 50 active projects underway at any one time.
Some of the company's other work rests on things like sensor research and emerging monitoring capabilities. "There are some design elements we are working on that will definitely reshape the way we think about data centers in general," he says. Some of those might even eventually turn into products.
Microsoft won't disclose the exact specs on the water towers or indeed many of its huge machines and pieces of other infrastructure that make the data center tick, but says it will likely do so in the future. Unlike Google, which has been mostly closed-mouth on its data centers, Microsoft has begun taking not only journalists, but also enterprise customers and software and hardware partners on tours in order to share best practices, technical detail and even carry out peer reviews. Manos has also been making the rounds speaking at data center shows.
"Historically, we've taken the approach that data centers are our competitive advantage, but 12 to 18 months after putting cool new stuff into place, it comes onto the market anyway," Manos says. It's Microsoft's business to sell more server software and more services, and that means divulging best practices and being transparent.
Microsoft customers have also told Microsoft that they need more information on what to measure and how, as well as what to implement and how. Manos believes there are far too many data center metrics floating about, and too little understanding of what to do with them. "The industry as a whole, we've got to get our act together," he says, adding that there's an increasing possibility of Federal legislation that may someday regulate data center efficiency in some way. He estimates that only 10% of companies have adopted any sort of capability to measure energy efficiency today, and that confusion reigns.
Take CADE, an energy efficiency metric McKinsey & Company recently unveiled. Manos says it’s a good metric, but it gives data center operators a number between one and 100 that's tough to decipher. Microsoft is working on a way to make CADE more meaningful by turning the raw numbers into a diagram of what it means and what a company needs to do to improve it.
Power is understandably a big issue in such huge data centers. Manos says power is increasingly costing more over the life of a server than the initial acquisition cost of that server. He believes there are many more revolutionary gains to be had in energy efficiency in just about every facet of the data center, including in electrical distribution, cooling, and patterns and practices.
In one sign Microsoft is serious about going green, the company instructed Turner Construction, which is building the data center in San Antonio, to save all of the old growth trees on the property. Only one, which is now nicknamed "the Manos tree" was within the building's original footprint, and stands alone in the parking lot.
Microsoft breaks its data centers into "classes" much like aircraft carriers. San Antonio is of the Quincy class, Microsoft's older and more traditional data centers. The Chicago class will be Microsoft's biggest data centers and will use self-contained shipping containers filled with servers as their unit of scale, while the Dublin class focuses entirely on energy efficiency. In the near future, the company plans to announce its newest data center and a new class of data centers while retiring another class.
The San Antonio data center will be ready to go by September, but right now Microsoft is just testing out its systems. The day I was there, it was doing heat testing with heat banks to make sure the air conditioning system could handle high heat output. In every case, Microsoft pushes its tests beyond the breaking point. And if you thought their data centers couldn't possibly get any bigger, they can: the new facility going up in Chicago is twice the size of San Antonio's data center.