Software // Information Management
Commentary
10/20/2004
03:22 PM
Ted Kemp
Ted Kemp
Commentary
Connect Directly
RSS
E-Mail
50%
50%

A Challenge For The Ages

Government agencies on the national, state and local levels want to build a means of effectively sharing and analyzing homeland security data. The full difficulty of that task will rival anything the government has ever accomplished.

Three years ago I talked to an IRS official about an ambitious project that would make it possible for people in the agency to instantly retrieve individual and business tax records. It was a conversation that taught me a lot about what a hugely difficult task data integration can be. Now, scores of government agencies on the national, state and local levels are trying to find ways to share and analyze data that relates to homeland security. If data integration at the IRS was a hard task, comprehensive integration and analysis of security data across all levels of government is going to be absolutely Herculean.

The database standardization project at the IRS, the official told me, involved pulling together information that resided in virtually every storage format known to mankind -- including mainframes, reel-to-reel tape, digital audio tapes and optical storage devices, among many others. The repositories used different means of data security, they were physically far-flung, and their ability to communicate with applications and each other was uneven or nonexistent. At the time, the agency projected it would take 10 years and $10 billion to complete its planned IT modernization initiatives.

Three years later, every time I see something on the news about efforts underway to make various security agencies work together and share information, I reflect back on that IRS interview. Homeland security involves many government departments and agencies, not just one. It involves many levels of government, not just the federal level. If the IRS's legacy systems were varied, the systems in place at the many entities that contribute to homeland security give new meaning to the word "diverse." I haven't even mentioned the varied formats of the data itself. And that's just the integration part: If the IRS's project involved actual analytics, it was trifling compared to what U.S. security imperatives will demand.

Some agencies, at least, are working to meet those demands. We hear about it piecemeal, but real efforts are underway. Law enforcement agencies around Washington are using wireless technology to integrate data from various jurisdictions. The FBI is deploying analysis software in its anti-terrorism efforts. And more money has been earmarked to secure the nation's ports. Not all the news is good: The Government Accountability Office said last month that the FBI lacks a coordinated plan to modernize its IT systems, and a report this month from the Department of Homeland Security's Inspector General office criticized the department for "not playing a lead role" in consolidating terrorist watchlist information.

The things we learn from the news media will inevitably make up only a tiny fraction of the whole story. That's as much a comment on the sheer scale of the job at hand as it is on the limitations of the media. As anyone familiar with data integration knows, there's a lot more to it than meets the eye. As anyone familiar with security knows, it's usually not a good idea to show your hand to your adversaries.

For all its infuriating inefficiency, arbitrary protocols and catty interagency rivalries, the U.S. government still managed to successfully partner with the private sector to build the Hoover Dam, the Liberty Ship fleet, the national highway system and the Apollo 11 lunar module. It absolutely is no exaggeration whatsoever to say that constructing a comprehensive means of accumulating, sharing and analyzing its myriad security data will be a task that rivals those. It will be carried out more quietly than the accomplishments that have preceded it. But it's got to be done.

Comment  | 
Print  | 
More Insights
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest September 24, 2014
Start improving branch office support by tapping public and private cloud resources to boost performance, increase worker productivity, and cut costs.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.