Data Deduplication: More Than Just A Feature - InformationWeek
IoT
IoT
Cloud // Cloud Storage
News
5/14/2008
02:15 PM
50%
50%

Data Deduplication: More Than Just A Feature

The major storage manufacturers have struggled to put together viable strategies for data reduction.

As data deduplication technology matured last year, I constantly heard industry analysts ask, "Isn't this just a feature?" The question implied that companies specifically in the data deduplication market were going to be erased by the larger storage manufacturers as they added deduplication capabilities to their products. It seemed logical, but it hasn't occurred. The major manufacturers have struggled to put together viable strategies for data reduction, and to some extent, it's really not in their best interest to reduce the amount of storage required.

For data deduplication to work well, it needs to be tightly integrated into the existing operating system of the disk itself. If you have a storage array OS whose source code is more than 3 years old, integrating a dramatically new way of placing data on that disk is going to be complex. The workaround to this problem is post-process deduplication, which individually analyzes each file to compare it with blocks of data that the system already has stored to determine redundancy--a time-consuming process (see story, "With Data Deduplication, Less Is More"). Another challenge with this method is that it creates two storage areas to manage: an area that's waiting to be examined for duplicates, and an area for data after it's been examined.

One of the benefits of deduplicated systems is that they store only unique data segments, replicating only new segments to remote locations. With the post-process method, you have to wait until the deduplication step is complete until data can be replicated. This can delay the update of the disaster-recovery site by six to 10 hours.

As a result, companies such as Data Domain, Permabit, and Diligent Technologies that started with data deduplication as a core part of their technology have a distinct advantage. Other vendors will have to make post-process data deduplication much more seamless, exit from deduplication altogether, or rewrite their code bases to support in-line deduplication.

chart: That Vulnerable Feeling: Is your company more vulnerable to malicious code attacks and security breaches than it was a year ago?

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of the Cloud Report
As the use of public cloud becomes a given, IT leaders must navigate the transition and advocate for management tools or architectures that allow them to realize the benefits they seek. Download this report to explore the issues and how to best leverage the cloud moving forward.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of November 6, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll