Comments
Big Data: No Hoarding Allowed
Threaded  |  Newest First  |  Oldest First
Lorna Garey
50%
50%
Lorna Garey,
User Rank: Author
7/7/2014 | 1:47:24 PM
Theory vs. reality
It's all great in theory. However, to save selectively requires effort and will -- data classification programs, someone to decide to delete X set and take the fall if it's needed someday, etc. Meanwhile, storage is cheap and getting cheaper.
Thomas Claburn
50%
50%
Thomas Claburn,
User Rank: Author
7/7/2014 | 1:55:27 PM
Re: Theory vs. reality
If only someone could convince the NSA of the merits of not hoarding data.
Doug Henschen
50%
50%
Doug Henschen,
User Rank: Strategist
7/7/2014 | 3:35:23 PM
Re: Theory vs. reality
Another opinion on big data from a self-interested vendor. Atkinson's "cost millions to data warehouse" perspective is a little dated. And the example he offers, tied to structured transactional data, is also not a very "big data" frame of reference.

The point of aggregating to the hour instead of the second is simple enough -- conventional wisdom, really. But this seems like a very conventional frame of reference focused on developing analytics based on recency, frequency, and monetary value. What about variable data types like clickstreams, log files, or social data? That's when data gets really big. It's not just a matter of collecting more of the same old data. 
zaious
50%
50%
zaious,
User Rank: Ninja
7/7/2014 | 11:19:21 PM
Re: Theory vs. reality
Playing it safe (storing all) is okay to a certain point -this keeps one risk free. However, it will be gone since the amount of data generation is increasing and it will require more to store. Humans are ,by nature, hoarders. At the same time it is tough to take the courage and hit the 'Delete' and confirm 'Yes'. 
StuartCarey
50%
50%
StuartCarey,
User Rank: Apprentice
7/8/2014 | 5:42:52 AM
Not Applicable to everything
Although a very interesting read, this is not always applicable to everything, such as senstive data sets (ie; Healthcare and NHS Where I work in the UK).

I tend to keep everything to allow us to do a previous year by year comparison on growth of a subject and more. Ie; 2 years ago there was no "SOP" in place, and now there is, this has been the change in data.

 


IT's Reputation: What the Data Says
IT's Reputation: What the Data Says
InformationWeek's IT Perception Survey seeks to quantify how IT thinks it's doing versus how the business really views IT's performance in delivering services - and, more important, powering innovation. Our results suggest IT leaders should worry less about whether they're getting enough resources and more about the relationships they have with business unit peers.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Government Tech Digest Oct. 27, 2014
To meet obligations -- and avoid accusations of cover-up and incompetence -- federal agencies must get serious about digitizing records.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
A roundup of the top stories and community news at InformationWeek.com.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.