Government // Big Data Analytics
Commentary
12/10/2013
10:01 AM
Joel Westphal
Joel Westphal
Commentary
Connect Directly
RSS
E-Mail

The War On Military Records

The era of big data has arrived on the battlefield and we need to find new ways to deal with it.

Comment  | 
Print  | 
Comments
Newest First  |  Oldest First  |  Threaded View
<<   <   Page 2 / 2
Joel5171
50%
50%
Joel5171,
User Rank: Apprentice
12/10/2013 | 3:14:51 PM
Re: Terabyte Limit Doesn't Compute
I cannot speak for rules and regulations from the National Archives. There are permanent records and temporary records. NARA usually does not want temporary records.

However, I think for the next conflict (and lets hope that day never comes) records issue there should be some discussion on keeping it all. But understand this opens up some serious issues for FOIA and eDiscovery.

Also, the 54 terabytes does not include the records from the Army, Navy and Air Force who collect their own service records. It also does not include other classified records.
Laurianne
50%
50%
Laurianne,
User Rank: Author
12/10/2013 | 2:55:11 PM
Re: Terabyte Limit Doesn't Compute
Doug makes a good point. Also, how did plentiful cloud storage options factor into the decision process?
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
12/10/2013 | 2:30:05 PM
Terabyte Limit Doesn't Compute
In the context of big data, 54 terabytes is no big deal. I don't get why the National Archives has cut off what they will save at 14 terabytes. Why not save it. You never know when some future revelation or future form of analysis renders that information relevant and useful. Geez, you can find 2 TB and 3 TB external hard drives for as little as $100. Why do we have to make any hard choices here. Save everything! History won't forgive you for repeting the same mistakes you described after the first Gulf War.
Joel5171
50%
50%
Joel5171,
User Rank: Apprentice
12/10/2013 | 12:15:20 PM
Re: Preserving Public Records
This is a great question! Given that the article had to be of a certain length a lot of what I wanted to say could not be included. This was one issue that I could not fit it.

A File Analysis tool is really just one answer to a question that requires multiple answers. For example, you still need an Electronic Records Management Application (ERMA) to ensure records are kept for as long as their disposition requires.

These tools help identify (and quickly) what and where the information is so you can then apply retention on them. They quickly assist the archivist or records manager in cleaning up those cluttered shared drives and SharePoint repositories that are out there.

I still hope to publish a more expansive article in either the American Archivist or Archivaria which really details how the entire operation to recover, preserve and then identify the records took place.
WKash
100%
0%
WKash,
User Rank: Author
12/10/2013 | 10:37:04 AM
Preserving Public Records
Thanks for sharing your experiences -- and the dilemma facing government records archivists.  Given the immense volumes of records generating daily, it is impossible to comprehend how individuals could sort through and manage all these records anymore.  My question is, are these content classification tools potentially breeding a false sense of security that they are identifying all the records that, by statute, must be preserved?

 
<<   <   Page 2 / 2
Skirting the Big Data Expertise Shortage
Skirting the Big Data Expertise Shortage
Federal departments and agencies have embraced big data in a big way, despite a shortage of trained and experienced workers, particularly data scientists. What tools and strategies are helping bridge the divide?
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest September 18, 2014
Enterprise social network success starts and ends with integration. Here's how to finally make collaboration click.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
The weekly wrap-up of the top stories from InformationWeek.com this week.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.