Flood, fire, solar flare, and crash by four-wheel-drive motor vehicle: These are the potential disasters that strike fear in the imaginations of data center operators, as the following accounts will show.
Jonathan Bryce, now executive director of the OpenStack Foundation, was the twenty-something founder of the Mosso Cloud in Dallas-Fort Worth when he found himself on the receiving end of such an incident on Dec. 18, 2009.
A diabetic driver in the vicinity of the Rackspace data center where Mosso was hosted had passed out behind the wheel of his SUV, crashing into a building housing the data center's electricity transformer equipment. Mosso was still running after the crash, but that was the start of a sequence of unlikely events that led to the service's outage.
How do you prepare for such an event in your disaster plan? "It's just one of those things you have to cope with as best you can," said Bryce.
This was a sentiment echoed last year by Robert von Woffradt, CIO for the State of Iowa, in a blog post after an unexpected fire in the state's primary data center. Survivors of the lower Manhattan office buildings and hospitals flooded by Hurricane Sandy in 2012 would agree.
Even if you think you're prepared for earthquake, flood and fire, when did you last worry adequately about the danger of solar flares? A powerful solar flare incident that could have disrupted electrical transmission systems missed the earth by a narrow margin in 2012. If the eruption had been only one week earlier, Earth would have been in the line of fire, Daniel Baker of the University of Colorado told NASA Science News in 2014. The flare's effects would have struck the earth's atmosphere, leading to heavy and unexpected voltage surges in electrical lines.
You might consider such a hazard as being extremely remote, but in 1859 a solar flare's disturbance known as the Carrington Event hit the earth and produced such large voltages that the wiring of telegraph offices sparked out of control, setting some offices on fire.
CIOs and data center managers who've been through a disaster have say that the best you can do is to prepare. "Test complete loss of systems at least once a year. No simulation; take them offline," advised Wolffradt, in a blog following the State of Iowa's crisis.
Check out our list of data center disasters -- from the scary to the outright outlandish -- and tell us about your own datacenter dramas in the comments section below.