A New York Times Sept. 22 article now causing heated conversation in the cloud computing community paints a gloomy picture of the average data center as a wastrel and energy glutton. In the eyes of the writer of the initial piece "The Cloud Factories: Power, Pollution and the Internet," data centers are prime suspects in the ongoing process of environmental degradation.
I am in total accord with the intent of the Times piece, seeking to make data centers more efficient. But I am dismayed at the casualties that the writer, James Glanz, inflicts as he goes about making his point. Surely, he realizes that the first part of the story describes an aging, mixed-system data center straight out of the 1980s or 1990s. Then he goes on to say that both the enterprise IT manager and cloud data center creator are culprits (two very different things lumped together). And, he concludes that the Internet user is an eager, witless accomplice to them both.
Either the Times underestimates the intelligence of its readers or it has lost touch with them. After all, it delights in publishing stories about the iPhone, and there are few drivers of data center construction that are equal to the use of smartphone apps. I don't remember reading about the dark side of iPhone 5 in all those iPhone 5 advance articles.
It's difficult, but what we should be trying to calculate is how to keep the economy growing with digital services, while protecting the environment.
Everyone is doing a lot more computing, as the story notes. But as we do so, the amount of electricity consumed per unit of computing is going down, which the story somehow misses. Nowhere does the Times address this salient point. Instead, it concludes we are doing a lot more computing and, therefore, we are all guilty of driving environmental degradation. If you're going to reform the world, you need to build a better soapbox than this.
[ Want to learn more about how Columbia Sportswear's cloud computing plans? See How Columbia Sportswear Will Survive Next Tsunami. ]
There's a lively discussion going on regarding these points at the New York Times' collection of related opinion columns, Big Data's Big Costs. "Is the convenience of cloud computing worth the energy and pollution it requires? Or should we wean off of 24/7 access?" are some of the questions asked.
Let me start with some contrary evidence. InformationWeek reported a year ago that the Environmental Protection Agency had predicted that between 2005 and 2010, the electricity consumed by data centers around the world would double, based on the rate of new construction. In 2011, the EPA had to back off that prediction because during that period, data center electricity consumption had increased by 36%, not 100%. The author of the study for the EPA speculated that the pace of data center construction had slowed, due to economic recession. I for one have driven around Silicon Valley looking for staked-out, but not-yet-built data centers; they're more scarce than Samsung handsets on Apple's Infinite Loop.
While some data centers were mothballed during the recession, most were built, and were much more efficient in using electricity than their predecessors. For example, PUE is the ratio of energy imported by a facility to the amount of power consumed by IT computing devices; it stands for Power Usage Effectiveness. The 1990s data center, with air conditioning pouring through a raised floor, and featuring a water-cooled mainframe at the center, had a PUE of 1.92 to 2. That means it consumed about twice as much power as the amount used in actual computing; cooling was the largest power consumer after computing itself.
Today, modern data centers built by Microsoft, Google, Facebook, or Yahoo have reduced that ratio to somewhere between 1.22 and 1.07--the latter mark set by Facebook's new Prineville, Ore., facility. This type of modern data center does so by delaying the stepping down of power line voltage until it's close to where servers and switches are running; that reduces the loss to resistance in the line. Instead of air conditioning, the modern data center uses ambient, outside air cooled by evaporation; the air is piped to the cool side of servers, blown into channels between baffles that steer it to the hottest components. Then it's collected in a hot aisle and flushed from the building. The process uses a fraction of the electricity of air conditioning.
The Times article doesn't mention PUE. It does cite a McKinsey study, commissioned by the newspaper, that found across several industries, the average server used only 6% to 12% of its available CPU cycles. To me, this describes the data center running one application per server, lest one application's activity overwrite the data of another (which causes a crash). In the past, InformationWeek has cited such a practice as utilizing 7% to 15% of the server, with the rest of the CPU cycles idled away. The practice reflects an IT manager's fear of downtime. At one time it was a realistic calculation that the business could afford the extra electricity, but it couldn't afford to go off the air for a few minutes or few hours.
But to some extent, the Times is busy pillorying a straw man. I don't know how to explain the McKinsey figures at this late date, but it's now 2012 and the times are a changin'. Maybe with his focus on the big picture, the writer missed the trend toward virtualization within the enterprise data center, where each server is loaded with 15 to 20 applications, or more. They run under lean hypervisor software, which manages CPU, memory, and I/O limits to avoid application conflicts. At sites where virtualization is used effectively, half or two-thirds of the number of physical servers formerly in use are now gone.
Columbia Sportswear, for example, during the last nine months has virtualized its main data center, moving it in the process from Portland, Ore., to Englewood, Colo. What had been 300 servers in Portland became 65 in Englewood. Those 65 are working harder than their predecessors; with each server running multiple applications, up to 80% of the CPU cycles are utilized, as opposed to the 7% to 15% range not so long ago. Michael Leeper, director of global technology, says he's looking for additional efficiencies.
Cloud data centers do the same thing, only more so. Instead of having air conditioned machines and IT staff in the data center, a cloud data center has a small operations crew in a command room running thousands of identical servers in a production facility. The machine space may be cooled to only 98 degrees. That's too hot for most people, but computers don't mind the higher temperature, saving electricity formerly used in cooling.
But the Times article rolls up the old enterprise data center into the new cloud data center without distinguishing between the two and offers up this quote, from an unnamed, "senior industry executive": the low rate of utilization is "an industry dirty secret. No one wants to be the first to say mea culpa. If we were a manufacturing industry, we'd be out of business."
That's an odd conclusion to reach. The only pollution cited is for operating unauthorized emergency diesel generators during an outage, for which Amazon Web Services and others have been cited in Virginia. Amazon paid a fine and got the required permits.
The Times indictment seems to be a broader one, that if any single industrial plant were responsible for the pollution caused by a data center, it would be out of business. But even this generous interpretation doesn't make sense. The electrical power utility is responsible for containing the pollution; the rate the data center pays covers its share of the cost. What is the Times trying to say? By virtue of the fact that it was built, the data center is guilty of something.
Larger Environmental Questions
Critics of the cloud on environmental grounds beg a much larger question. To reach the conclusions that Glanz does, they must assume running more cloud data centers is inherently wasteful and wrong. They seem to assume much of the activity these data centers support is trivial and of little social value; the activity's main result is to incur more environmental damage.
This is a Luddite view of what's happening in the post-PC, digital era. As more of the economy is digitized, you can argue that it relies less on the consumption of materials and movement of goods and more on digital services executed electronically. There are often favorable trade-offs to the environment in these transactions.
For example, when Amazon.com or any online supplier engages a customer, that means someone hasn't gotten into a car and driven to a store. If the site visitor makes a purchase, it must be delivered, but UPS and other delivery services consolidate many shippers' packages into one truck route. That's a favorable alternative to multiple trips. Isn't the cloud is functioning efficiently there?
Yes, people are engaged in playing games in the cloud, following Twitter, and checking their bank accounts. But cloud services also help buyers sift through information when making a major purchase, such as a car, reducing the time spent going from dealership to dealership. Likewise, few people drive to travel agents offices to plan a trip anymore. They move digital bits around instead.
The Department of Motor Vehicles distributing an electronic form saves thousands of consumers time and money, while reducing reliance on paper produced from trees. Electronic books arrive at their destination without any motor vehicle assistance. Who's done a full energy audit that ends up concluding the cloud data center is an energy wastrel in these exchanges? I don't see one in the Times.
On a larger scale, enterprises used to have to establish a disaster recovery site with hardware and software that closely mimicked the main data center. Now they can move digital bits--instead of equipment--in the form of virtual machine files that travel over the wire instead. Columbia Sportswear is an example of a company establishing back up virtual machines in the cloud. It's planning to establish the capability to move the virtual machine files around, wherever they're needed, after a natural disaster occurs. Such a maneuver was needed last year when the tsunami struck northeastern Japan, causing Columbia's Tokyo data center to shut down.
The Times article also skips discussion of data center consolidation--combining multiple data center physical sites into one site--which represents one of the biggest trends in enterprise and government IT in recent years. Virtualization and cloud technologies have aided in that consolidation work.
In many instances, it's better for the environment that the cloud data center exists. It's acting as a substitute for higher energy-consuming, physical activity. Nevertheless, the assumptions aired in the Times article, erroneous or not, are sure to become broader societal concerns, if four years from now the Midwest is still caught in a withering drought and the world's food supplies are crashing. In the face of environmental disaster, whether a particular data center functions efficiently or not is probably going to be moot. Both business and consumers will find that their data centers have become government-regulated utilities and their use is being strictly rationed--to cut emissions of course.
To avoid such an outcome and keep a rational discourse going, best practices for efficient data center operation should be debated and implemented. Energy hogs need to be retired, perhaps ahead of life expectancy. A case needs to be built that an efficient enterprise data center, working in concert with a cloud facility, can use the new hybrid cloud model of computing to expel some of the worst over-provisioning practices of yesteryear's IT organizations.
Data centers are not the problem. They're part of the solution. But we have a ways to go to establish that.