I'd like to reemphasize a point I've been making for a while about data retention: As costs go down, the wisdom of keeping detailed data goes up. I'd go so far as to say that every piece of data generated by a human being should be preserved and kept online...
I'd like to reemphasize a point I've been making for a while about data retention:
As costs go down, the wisdom of keeping detailed data goes up. I'd go so far as to say that every piece of data generated by a human being should be preserved and kept online, legal and privacy considerations permitting.* Most forms of capital-, labor-, and/or location-based competitive advantage being commoditized and/or globalized away. But information remains a unique corporate asset. Don't discard it lightly.
*Unless there's an explicit law mandating data destruction, legal considerations should permit. The idea "Let's destroy something of irreplaceable value today, against the possibility we might be brought to judgment tomorrow" is both morally and pragmatically weird. Privacy, however, may be a different matter.That applies to the structured/tabular kinds of data I tend to focus on in this blog. It applies even more to anything that's like a document (or email, instant message, whatever) somebody has taken the trouble to place into words. A top document-oriented archiving analyst (and my good friend), David Ferris, quiteagrees. As David puts it:
I think we'll end up archiving everything, except egregious garbage like spam:
It's too hard to get users to conform to policy.
Automated methods of capturing a human-understandable policy, for example "tax records," are too hard to implement through automatic filters. The filters are too inaccurate.
It's impractical to get users to classify everything, and automatic classification is too crude.
You never know what you might want later. Stuff you think you won't want now may end up being very useful.
The cost of storage is trivial when looked at on a per-user basis.
In particular, I think information destruction is a crude instrument for the protection of privacy, wasteful at best, and likely to be vigorously resisted by governments and large businesses. For example:
Businesses are increasingly subject to retention-oriented compliance regulation. Your lawyers may want you to destroy information that could be used to sue you, but governments won't let you.
Besides, archiving technologies are getting ever more cost-effective.I'd like to reemphasize a point I've been making for a while about data retention: As costs go down, the wisdom of keeping detailed data goes up. I'd go so far as to say that every piece of data generated by a human being should be preserved and kept online...
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.
InformationWeek Must Reads Oct. 21, 2014InformationWeek's new Must Reads is a compendium of our best recent coverage of digital strategy. Learn why you should learn to embrace DevOps, how to avoid roadblocks for digital projects, what the five steps to API management are, and more.