7 Data Center Disasters You'll Never See Coming - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Cloud
News
6/7/2015
12:06 PM
Charles Babcock
Charles Babcock
Slideshows
Connect Directly
Twitter
RSS
E-Mail
100%
0%

7 Data Center Disasters You'll Never See Coming

These are the kinds of random events that keep data center operators up at night. Is your disaster recovery plan prepared to handle these freak accidents?
Previous
1 of 9
Next

(Image: toxawww via iStockphoto)

(Image: toxawww via iStockphoto)

Flood, fire, solar flare, and crash by four-wheel-drive motor vehicle: These are the potential disasters that strike fear in the imaginations of data center operators, as the following accounts will show.

Jonathan Bryce, now executive director of the OpenStack Foundation, was the twenty-something founder of the Mosso Cloud in Dallas-Fort Worth when he found himself on the receiving end of such an incident on Dec. 18, 2009.

A diabetic driver in the vicinity of the Rackspace data center where Mosso was hosted had passed out behind the wheel of his SUV, crashing into a building housing the data center's electricity transformer equipment. Mosso was still running after the crash, but that was the start of a sequence of unlikely events that led to the service's outage.

How do you prepare for such an event in your disaster plan? "It's just one of those things you have to cope with as best you can," said Bryce.

This was a sentiment echoed last year by Robert von Woffradt, CIO for the State of Iowa, in a blog post after an unexpected fire in the state's primary data center. Survivors of the lower Manhattan office buildings and hospitals flooded by Hurricane Sandy in 2012 would agree.

Even if you think you're prepared for earthquake, flood and fire, when did you last worry adequately about the danger of solar flares? A powerful solar flare incident that could have disrupted electrical transmission systems missed the earth by a narrow margin in 2012. If the eruption had been only one week earlier, Earth would have been in the line of fire, Daniel Baker of the University of Colorado told NASA Science News in 2014. The flare's effects would have struck the earth's atmosphere, leading to heavy and unexpected voltage surges in electrical lines.

You might consider such a hazard as being extremely remote, but in 1859 a solar flare's disturbance known as the Carrington Event hit the earth and produced such large voltages that the wiring of telegraph offices sparked out of control, setting some offices on fire.

CIOs and data center managers who've been through a disaster have say that the best you can do is to prepare. "Test complete loss of systems at least once a year. No simulation; take them offline," advised Wolffradt, in a blog following the State of Iowa's crisis.

Check out our list of data center disasters -- from the scary to the outright outlandish -- and tell us about your own datacenter dramas in the comments section below.

Charles Babcock is an editor-at-large for InformationWeek and author of Management Strategies for the Cloud Revolution, a McGraw-Hill book. He is the former editor-in-chief of Digital News, former software editor of Computerworld and former technology editor of Interactive ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Previous
1 of 9
Next
Comment  | 
Print  | 
More Insights
Comments
Oldest First  |  Newest First  |  Threaded View
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
6/8/2015 | 12:48:52 PM
When the fire fighting system gets triggered by accident....
DanaRothlock, Yes, part of the problem of disaster preparedness is preventing the fire fighting system, especially when it's triggered by accident, from destroying what it's supposed to save. There's been no easy answer for years. Halon was meant to prevent water damage to the equipment. Sprinklers, on the other hand, prevent Halon damage. It's a fool's bargain with fate.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
6/8/2015 | 9:07:39 PM
Move the backup site further away!
Dave Kramer, yes, it's a good idea to move the backup data center to a different site. But Hurricane Sandy told us just how far away that second site might have to be. Moving it across town or across the state might not have been enough in that case. With Sandy, disaster recovery specialist Sungard, had flood waters lapping at the edges of its parking lots on the high ground in N.J. The advent of disaster recovery based on virtual machines makes it more feasible to move recovery to a distant site (but still doesn't solve all problems).
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
6/11/2015 | 3:02:16 PM
Diesel fuel stored at NYC data centers reduced by 9/11
KBartle, what about this? One of the unreported aspects of the Hurricane Sandy disaster, when New York and many places along the East Coast went dark, was that every data center in the city had a limited supply of diesel fuel on premises. That was due to new regulations, I believe from a former mayor's office after 9/11, that the flamable liquids stored inside an office building must be reduced. In some cases, that made the investment in generators irrelevant. Public transit was down, city streets were clogged and fuel delivery trucks had great difficulty getting through. There goes the disaster recovery plan.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
6/11/2015 | 3:15:00 PM
A narrow margin separates "chilled" from "too hot"
In no. 6, Outage by SUV, a commenter on Lew Moorman's blog post noted that a data center has about five minutes between the loss of its chillers and the start of equipment overheating. Does anyone know, is the margin really that narrow? I understand that computer equipment can operate at up to 100 degrees OK, but after that overheating starts to get dicey.
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

News
Pandemic Responses Make Room for More Data Opportunities
Jessica Davis, Senior Editor, Enterprise Apps,  5/4/2021
Slideshows
10 Things Your Artificial Intelligence Initiative Needs to Succeed
Lisa Morgan, Freelance Writer,  4/20/2021
News
Transformation, Disruption, and Gender Diversity in Tech
Joao-Pierre S. Ruth, Senior Writer,  5/6/2021
White Papers
Register for InformationWeek Newsletters
2021 State of ITOps and SecOps Report
2021 State of ITOps and SecOps Report
This new report from InformationWeek explores what we've learned over the past year, critical trends around ITOps and SecOps, and where leaders are focusing their time and efforts to support a growing digital economy. Download it today!
Video
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Slideshows
Flash Poll