Power fluctuations Tuesday set off a chain reaction that took down a data center that hosts Craigslist, Six Apart's blog sites, Technorati, Yelp, and others.
A power outage in downtown San Francisco and south of the city Tuesday affected some 40,000 Pacific Gas & Electric customers directly and millions of Internet users from all over who were unable to access several popular Web sites that went offline.
AdBrite.com, Craigslist.org, Six Apart's blog sites (LiveJournal.com, TypePad.com, Vox.com), Technorati.com, and Yelp.com were all inaccessible for part of the day. Other sites, including CurrentTV.com, RedEnvelope.com, and SecondLife.com, also experienced problems.
The power failure began at about 1:30 p.m. Pacific time, according to PG&E, and power was restored to all customers within two hours.
According to a PG&E spokesperson, power fluctuations began triggering safety breakers, resulting in intermittent power spikes. Those fluctuations appear to have damaged an underground transformer that exploded about 2:30 pm. The company is still investigating the caused of the power fluctuation and expects to issue an update later today.
Many of affected Web sites are hosted at 365 Main, which "develops and operates the world's finest data centers." Like all serious data centers, 365 Main has backup generators for just such an occasion. Those backup generators didn't work as planned, however.
"While backup electrical infrastructure is installed in the facility to defend against power surges, an initial investigation has revealed that certain 365 Main back-up generators did not start when the initial power surge hit the building," marketing VP Miles Kelly said in a statement. "On-site facility engineers responded and manually started affected generators allowing stable power to be restored at approximately 2:34 p.m. across the entire facility."
It took several more hours for Six Apart to get its sites up and running again, according to a company spokesperson.
Kelly said that 365 Main is still investigating why the backup generated did not activate and that the company intends to take steps to prevent this from happening again. He also said the company's diesel generators would continue to operate until the cause of the backup generator failure had been identified.
The failure of the backup generators may be a consequence of 365 Main's reliance on a flywheel uninterruptible power supply, a kinetic energy technology considered to be more "green" than a battery-driven UPS. On the Data Center Knowledge blog, technology journalist Rich Miller reports, "Some customers speculated about a flywheel issue."
At InformationWeek's San Francisco office, where computer monitors went dark several times on Tuesday afternoon, the power failure pointed out one advantage of laptop PCs over desktop systems: Laptop batteries serve as an emergency power supply, preventing the data loss that may be experienced by desktop systems when wall outlets go dead.
How Enterprises Are Attacking the IT Security EnterpriseTo learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
IT Strategies to Conquer the CloudChances are your organization is adopting cloud computing in one way or another -- or in multiple ways. Understanding the skills you need and how cloud affects IT operations and networking will help you adapt.