Data Management // Big Data Analytics
12:44 PM
Connect Directly
Core System Testing: How to Achieve Success
Oct 06, 2016
Property and Casualty Insurers have been investing in modernizing their core systems to provide fl ...Read More>>

Are You A Data Hoarder?

Big data craze inspires some IT managers to save every possible bit of data. Bad idea, says an industry practitioner's group.

9 Bandwidth Hogs: Reality Vs. Myth
9 Bandwidth Hogs: Reality Vs. Myth
(click image for larger view and for slideshow)

You've heard it before: There's gold buried deep in your organization's data -- valuable insights that a little mining can bring to the surface.

True? In some cases, yes, but a hoarding impulse can backfire too. In fact, enterprises that implement big data analytics platforms -- applying complex mathematical algorithms to unlock new trends and customer insights -- can easily turn into information pack rats, says the Compliance, Governance and Oversight Council (CGOC), a forum of about 2,300 legal, IT, records and information management professionals from business and government agencies.

"We're starting to hear some CTOs say, 'Well, we don't want to delete our data if we can mine value out of it,'" said Jake Frazier, CGOC executive director who's also an attorney who works at IBM, in a phone interview with InformationWeek. "I think that's a false dichotomy. Once you delete data that's stale, the algorithms actually function much better from an analytics standpoint. Leaving stale data can actually skew the algorithms towards older facts."

[ How do you decide which data has real value? Here's how to build a smart data strategy: Turning Raw Data Into Smart Data. ]

The CGOC claims that information hoarding costs enterprises thousands or even millions of dollars annually in unneeded infrastructure and storage costs, not to mention legal fees for reviewing documents that should be trashed. It also lowers productivity as workers sift through vast stockpiles of digital data to find information they need.

According to CGOC survey results, organizations on average need to archive about 2-3% of their data for legal hold, 5-10% to meet regulatory requirements, and 25% for business analysis and insights.

But there's a problem here: Enterprises often aren't sure what information they need to keep, or where to find it. "A lot of organizations figured this out eight or 10 years ago. The logic isn't hard to understand, but the devil is in the details," said Frazier. "Out of paralysis, they tend to save everything."

Enterprises, for instance, often warehouse vast numbers of backup tapes, most of which have data that could (and should) be eliminated. "IT is most keenly aware of the problem. They go to legal and records and say, 'Hey, we've got to get rid of stuff, what can we get rid of?'" Frazier said. "But they don't end up with the instructions they need. Then they have no choice but to solve the problem by containment."

This often means moving data around. Frazier uses this analogy when explaining the issue to his fellow attorneys: "When my kids don't want to eat their green vegetables, they move them around the plate, behind things and under things," he said. "It makes it look like there's more empty plate there, but they're not actually solving the problem."

Things appear to be improving, however. "The technology now exists to scan those tapes without reloading them and retrieving the backups. One of my IBM clients, one of the largest banks in the world, is scanning hundreds of thousands of tapes," said Frazier. This important step will allow the bank to delete data that doesn't have legal hold or regulatory requirements.

A growing number of privacy regulations in the U.S. may pose a challenge for American corporations, which must navigate stricter European privacy rules as well. "In some cases we see a conflict," said Frazier. For instance, U.S. rules may require a company to retain certain information for six years, whereas a European nation may require the business to delete the same data after three years.

"That's another reason why this isn't a problem where you stick your head in the sand," Frazier noted.

Attend Interop Las Vegas, May 6-10, and attend the most thorough training on Apple Deployment at the NEW Mac & iOS IT Conference. Use Priority Code DIPR02 by March 2 to save up to $500 off the price of Conference Passes. Join us in Las Vegas for access to 125+ workshops and conference classes, 350+ exhibiting companies, and the latest technology. Register for Interop today!

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.