Big Data // Big Data Analytics
12:44 PM
Connect Directly
Repost This

Are You A Data Hoarder?

Big data craze inspires some IT managers to save every possible bit of data. Bad idea, says an industry practitioner's group.

9 Bandwidth Hogs: Reality Vs. Myth
9 Bandwidth Hogs: Reality Vs. Myth
(click image for larger view and for slideshow)

You've heard it before: There's gold buried deep in your organization's data -- valuable insights that a little mining can bring to the surface.

True? In some cases, yes, but a hoarding impulse can backfire too. In fact, enterprises that implement big data analytics platforms -- applying complex mathematical algorithms to unlock new trends and customer insights -- can easily turn into information pack rats, says the Compliance, Governance and Oversight Council (CGOC), a forum of about 2,300 legal, IT, records and information management professionals from business and government agencies.

"We're starting to hear some CTOs say, 'Well, we don't want to delete our data if we can mine value out of it,'" said Jake Frazier, CGOC executive director who's also an attorney who works at IBM, in a phone interview with InformationWeek. "I think that's a false dichotomy. Once you delete data that's stale, the algorithms actually function much better from an analytics standpoint. Leaving stale data can actually skew the algorithms towards older facts."

[ How do you decide which data has real value? Here's how to build a smart data strategy: Turning Raw Data Into Smart Data. ]

The CGOC claims that information hoarding costs enterprises thousands or even millions of dollars annually in unneeded infrastructure and storage costs, not to mention legal fees for reviewing documents that should be trashed. It also lowers productivity as workers sift through vast stockpiles of digital data to find information they need.

According to CGOC survey results, organizations on average need to archive about 2-3% of their data for legal hold, 5-10% to meet regulatory requirements, and 25% for business analysis and insights.

But there's a problem here: Enterprises often aren't sure what information they need to keep, or where to find it. "A lot of organizations figured this out eight or 10 years ago. The logic isn't hard to understand, but the devil is in the details," said Frazier. "Out of paralysis, they tend to save everything."

Enterprises, for instance, often warehouse vast numbers of backup tapes, most of which have data that could (and should) be eliminated. "IT is most keenly aware of the problem. They go to legal and records and say, 'Hey, we've got to get rid of stuff, what can we get rid of?'" Frazier said. "But they don't end up with the instructions they need. Then they have no choice but to solve the problem by containment."

This often means moving data around. Frazier uses this analogy when explaining the issue to his fellow attorneys: "When my kids don't want to eat their green vegetables, they move them around the plate, behind things and under things," he said. "It makes it look like there's more empty plate there, but they're not actually solving the problem."

Things appear to be improving, however. "The technology now exists to scan those tapes without reloading them and retrieving the backups. One of my IBM clients, one of the largest banks in the world, is scanning hundreds of thousands of tapes," said Frazier. This important step will allow the bank to delete data that doesn't have legal hold or regulatory requirements.

A growing number of privacy regulations in the U.S. may pose a challenge for American corporations, which must navigate stricter European privacy rules as well. "In some cases we see a conflict," said Frazier. For instance, U.S. rules may require a company to retain certain information for six years, whereas a European nation may require the business to delete the same data after three years.

"That's another reason why this isn't a problem where you stick your head in the sand," Frazier noted.

Attend Interop Las Vegas, May 6-10, and attend the most thorough training on Apple Deployment at the NEW Mac & iOS IT Conference. Use Priority Code DIPR02 by March 2 to save up to $500 off the price of Conference Passes. Join us in Las Vegas for access to 125+ workshops and conference classes, 350+ exhibiting companies, and the latest technology. Register for Interop today!

Comment  | 
Print  | 
More Insights
InformationWeek Elite 100
InformationWeek Elite 100
Our data shows these innovators using digital technology in two key areas: providing better products and cutting costs. Almost half of them expect to introduce a new IT-led product this year, and 46% are using technology to make business processes more efficient.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.