Though state agencies have resumed IT operations, the Virginia Information Technologies Agency and Northrop Grumman, which has apologized for a more than weeklong failure, are still working to recover data.
Image Gallery: Government's 10 Most Powerful Supercomputers
IT operations in Virginia are more or less back to normal after an unprecedented network failure, but the state is still working "around the clock" to recover data, state CIO Sam Nixon said. He said that data loss has been "minimal" but that hopefully all of it will be recovered once recovery operations are completed.
Nixon also is agency head at Virginia Information Technologies Agency (VITA), which was created in 2003 to oversee and modernize the state's IT operations. A more than week-long disruption in IT operations and services for 27 state government agencies ended Thursday. The ordeal started last Wednesday when an EMC DMX-3 storage area network (SAN) went down, taking 13 percent of the state's file servers with it.
By Friday, VITA and Northrop Grumman, which partners with VITA to maintain IT networks for the state, had repaired the SAN. However, it took until Tuesday for nearly all affected agencies to be back online and to business as usual. Even then, the Virginia Department of Motor Vehicles, department of taxation and state board of elections still experienced disruptions. Officials said the complexity and size of their databases caused recovery to take longer for these agencies.
On Thursday Northrop, which signed a controversial $2.3 billion contract with VITA in 2005, formally apologized for the failure and said it would do everything it could to avoid such a scenario in the future. "Northrop Grumman deeply regrets the disruption and inconvenience this has caused state agencies and Virginia citizens," said Linda A. Mills, corporate vice president and president, Northrop Grumman Information Systems, in a statement. She said that while the "disruption to the business of state government has ended," the firm's investigation into the problem has not. "Our efforts to examine the factors that contributed to the initial problem and the lengthy restoration of service is just beginning," Mills said.
Northrop also plans to reimburse Virginia for "the reasonable costs of an assessment" of the problem, "as it is an essential and responsible measure that will strengthen our ability to protect against future issues," she said. Revisions to Northrop's own plan to improve process and response time for restoring agency operations also will occur as a result of the outage, Mills said.
Mills also acknowledged criticism to the state's decision to partially outsource IT operations to a private company, which has been widespread since its inception. The outage was the second major one since the two joined forces to handle IT operations for Virginia. "The partnership has experienced its share of obstacles, not unusual with large transformation programs," Mills said. "But with this modernized system Virginia and her citizens should find themselves years ahead of other states with the service provided by its IT infrastructure."
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.
Join us for a roundup of the top stories on InformationWeek.com for the week of December 14, 2014. Be here for the show and for the incredible Friday Afternoon Conversation that runs beside the program.