Nasdaq switched over to backup generators at its primary data center in Connecticut within minutes after losing power at 4:12 p.m., causing minimal disruptions in after-hours trading. In addition, the exchange switched control of some 100 core applications to a fully redundant data center in Maryland. "Other than the fact that we were on backup diesel generators, our systems were normal," senior VP Anna Ewing says. Commercial power was restored to the primary data center at 10 p.m., but it remained on backup power until 11 p.m. as an additional safety measure. By 4 a.m., all member firms were able to connect to the system, ensuring a normal trading day on Friday.
A well-tested and thought-out business-continuity plan is critical in today's high-tech marketplaces. "We are an electronic stock market, so contingency planning is key to us," Ewing says.
At the New York Stock Exchange, commercial power wasn't restored until 6 a.m. but systems were running on backup power and the exchange never considered delaying its opening Friday. Had the outage occurred during trading hours, trading would have been suspended--but backup data centers would have ensured that no trading data was lost, a spokesman says.
The Federal Reserve Bank of New York was able to perform all of its central banking functions fully despite the loss of power in its lower Manhattan headquarters.
Although individual ATMs operated by Citibank, J.P. Morgan Chase, Bank of New York and other major institutions were knocked out of service, the ATM networks continued to operate. At Bank of New York, backup power facilities enabled workers at the company's Wall Street headquarters to remain at their posts--many of them overnight--while workers at other Manhattan offices were streaming into the streets, spokesman Jeep Bryant says. The company maintains redundant data centers in Florida, New Jersey, and upstate New York.
The only exchange that didn't resume normal operations was the American Stock Exchange, which didn't reopen until 3:45 p.m. Friday. The issue wasn't electricity--it had an ample supply from backup generators--but steam. Amex depends on steam purchased from the local utility, Consolidated Edison Co., to drive the air conditioners that cool its vast trading floor. At 7 p.m. Thursday, Con Edison assured Amex that steam would be available the next morning. But Con Edison had neglected to mention that, like Amex, many other businesses in the area also use steam for their air conditioners--but unlike Amex, these other businesses lacked electrical power. "If Con Ed were to send steam through the network, all of those units would have been flooded because their air-conditioner units weren't turned on," says Amex senior VP Bob Rendine.
By the time Con Edison notified Amex of this situation, it was too late to move to the backup facility. "Because the decision to move to that facility needed to be made by 7 p.m. the evening of the blackout, it wasn't feasible to do so by the time we were informed of the steam-delivery issue on Friday morning," Rendine says.
With the help of New York's Office of Emergency Management, Amex procured a portable boiler to produce steam, enabling it to reopen for trading shortly before Friday's official close. Opening even for a short time allowed it to conduct closeouts on options, equities, and exchange-traded funds on an expiration Friday.
Rendine says the lesson is that no plan, however well thought-out, is foolproof. "Nobody planned for a contingency of 50 million people losing power," he says. "If you spent all your time planning for contingencies, you'd be in the business of planning for contingencies instead of running a stock market."