Are You A Data Hoarder?

9 Bandwidth Hogs: Reality Vs. Myth

(click image for larger view and for slideshow)

You've heard it before: There's gold buried deep in your organization's data -- valuable insights that a little mining can bring to the surface.

True? In some cases, yes, but a hoarding impulse can backfire too. In fact, enterprises that implement big data analytics platforms -- applying complex mathematical algorithms to unlock new trends and customer insights -- can easily turn into information pack rats, says the Compliance, Governance and Oversight Council (CGOC), a forum of about 2,300 legal, IT, records and information management professionals from business and government agencies.

"We're starting to hear some CTOs say, 'Well, we don't want to delete our data if we can mine value out of it,'" said Jake Frazier, CGOC executive director who's also an attorney who works at IBM, in a phone interview with InformationWeek. "I think that's a false dichotomy. Once you delete data that's stale, the algorithms actually function much better from an analytics standpoint. Leaving stale data can actually skew the algorithms towards older facts."

[ How do you decide which data has real value? Here's how to build a smart data strategy: Turning Raw Data Into Smart Data. ]

The CGOC claims that information hoarding costs enterprises thousands or even millions of dollars annually in unneeded infrastructure and storage costs, not to mention legal fees for reviewing documents that should be trashed. It also lowers productivity as workers sift through vast stockpiles of digital data to find information they need.

According to CGOC survey results, organizations on average need to archive about 2-3% of their data for legal hold, 5-10% to meet regulatory requirements, and 25% for business analysis and insights.

But there's a problem here: Enterprises often aren't sure what information they need to keep, or where to find it. "A lot of organizations figured this out eight or 10 years ago. The logic isn't hard to understand, but the devil is in the details," said Frazier. "Out of paralysis, they tend to save everything."

Enterprises, for instance, often warehouse vast numbers of backup tapes, most of which have data that could (and should) be eliminated. "IT is most keenly aware of the problem. They go to legal and records and say, 'Hey, we've got to get rid of stuff, what can we get rid of?'" Frazier said. "But they don't end up with the instructions they need. Then they have no choice but to solve the problem by containment."

This often means moving data around. Frazier uses this analogy when explaining the issue to his fellow attorneys: "When my kids don't want to eat their green vegetables, they move them around the plate, behind things and under things," he said. "It makes it look like there's more empty plate there, but they're not actually solving the problem."

Things appear to be improving, however. "The technology now exists to scan those tapes without reloading them and retrieving the backups. One of my IBM clients, one of the largest banks in the world, is scanning hundreds of thousands of tapes," said Frazier. This important step will allow the bank to delete data that doesn't have legal hold or regulatory requirements.

A growing number of privacy regulations in the U.S. may pose a challenge for American corporations, which must navigate stricter European privacy rules as well. "In some cases we see a conflict," said Frazier. For instance, U.S. rules may require a company to retain certain information for six years, whereas a European nation may require the business to delete the same data after three years.

"That's another reason why this isn't a problem where you stick your head in the sand," Frazier noted.

Attend Interop Las Vegas, May 6-10, and attend the most thorough training on Apple Deployment at the NEW Mac & iOS IT Conference. Use Priority Code DIPR02 by March 2 to save up to $500 off the price of Conference Passes. Join us in Las Vegas for access to 125+ workshops and conference classes, 350+ exhibiting companies, and the latest technology. Register for Interop today!