The Case for Retaining Everything - InformationWeek
IoT
IoT
Software // Information Management
Commentary
4/6/2010
06:13 PM
Curt Monash
Curt Monash
Commentary
50%
50%

The Case for Retaining Everything

I'd like to reemphasize a point I've been making for a while about data retention: As costs go down, the wisdom of keeping detailed data goes up. I'd go so far as to say that every piece of data generated by a human being should be preserved and kept online...

I'd like to reemphasize a point I've been making for a while about data retention:

As costs go down, the wisdom of keeping detailed data goes up. I'd go so far as to say that every piece of data generated by a human being should be preserved and kept online, legal and privacy considerations permitting.* Most forms of capital-, labor-, and/or location-based competitive advantage being commoditized and/or globalized away. But information remains a unique corporate asset. Don't discard it lightly. *Unless there's an explicit law mandating data destruction, legal considerations should permit. The idea "Let's destroy something of irreplaceable value today, against the possibility we might be brought to judgment tomorrow" is both morally and pragmatically weird. Privacy, however, may be a different matter.That applies to the structured/tabular kinds of data I tend to focus on in this blog. It applies even more to anything that's like a document (or email, instant message, whatever) somebody has taken the trouble to place into words. A top document-oriented archiving analyst (and my good friend), David Ferris, quite agrees. As David puts it:

I think we'll end up archiving everything, except egregious garbage like spam:

  • It's too hard to get users to conform to policy.
  • Automated methods of capturing a human-understandable policy, for example "tax records," are too hard to implement through automatic filters. The filters are too inaccurate.
  • It's impractical to get users to classify everything, and automatic classification is too crude.
  • You never know what you might want later. Stuff you think you won't want now may end up being very useful.
  • The cost of storage is trivial when looked at on a per-user basis.
In particular, I think information destruction is a crude instrument for the protection of privacy, wasteful at best, and likely to be vigorously resisted by governments and large businesses. For example:
  • Businesses are increasingly subject to retention-oriented compliance regulation. Your lawyers may want you to destroy information that could be used to sue you, but governments won't let you.
  • Information about individuals' web surfing is being retained, under law, so that they may be fingered later for pornography consumption or illegal file sharing. I deplore some of the ways web-surfing data can be and is being used, and want laws passed to rein them in. But the retention will happen.
  • Marketers want all that data. Duh.
  • Electronic health records are coming -- slowly, but they'll get here some day.
Besides, archiving technologies are getting ever more cost-effective.I'd like to reemphasize a point I've been making for a while about data retention: As costs go down, the wisdom of keeping detailed data goes up. I'd go so far as to say that every piece of data generated by a human being should be preserved and kept online...

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of the Cloud Report
As the use of public cloud becomes a given, IT leaders must navigate the transition and advocate for management tools or architectures that allow them to realize the benefits they seek. Download this report to explore the issues and how to best leverage the cloud moving forward.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of November 6, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll