How To Protect The Big Data Archive - InformationWeek
Cloud // Cloud Storage
10:49 AM
George Crump
George Crump

How To Protect The Big Data Archive

Your big data disaster recovery strategy must consider size, cost, and the unpredictable, but inevitable, need to access old data.

Protecting a big data archive is different than protecting big data analytics--or it should be. While both types of big data are, well big, big data archives typically have the larger capacity. Full backups are not an option (too much data), and a disk-only backup strategy may not be realistic (too expensive). Big data archives must have data protection embedded into the architecture, as the data sets are too large to run a separate process.

As we discussed in a recent column big data archive environments are storing dozens of petabytes of content, often video, audio, or images. That content can sit idle for years and then a portion of it becomes active, triggered by some event, for a period of time. A common example: When celebrities die or get into trouble, video and photos from their past (that were infrequently accessed) become heavily requested. The size of these environments makes backup almost impossible, but the likelihood of accessing that data makes protection a top requirement.

The challenge presented by the data set's size and access pattern is such that creating a disk-only archive store may not be practical. The challenge is not one of capacity, as we discuss in our article "Is Your File Server Choking." Object-based storage from cloud infrastructure suppliers has eliminated the key scaling issues facing traditional file systems, adding near-infinite capacity and dealing with trillions of files. The challenge is the cost associated with providing a near-infinite capacity, disk-based storage system and backing it up via replication to another disk-based, near-infinite capacity system.

As a result, tape has a role, and potentially a prominent one, in big data archives. Archive file systems, like those offered by members of the Active Archive Alliance, have the capability to support tape as an integral part of the environment. What this means is that a large percentage of the active data sets can still be kept on disk for instant access, but can also be copied to a tape or two upon creation. The second tape can be used for backup and disaster recovery. This means "backup" happens as the data is created or modified, not as a single, separate, nightly process. As the disaster-recovery tape fills, it can be moved to a secure offsite location. It also means that tape can be the only location of that data, eventually scrubbing the data from disk.

This is not an "abandon disk" recommendation; it is a recommendation to be realistic. We suggest that you put as much data on disk as makes sense and can be afforded. Disk is needed for the data that will become hot, driven by the next world event. It is also needed for recently created data, since that is the most likely to be accessed.

Tape is needed to back up all the information and make sure the petabytes of old information that won't be accessed is cost-effectively stored. As we discuss in our article "Comparing LTO-6 to Scale Out Storage for Long-Term Retention," linear tape open (LTO) now has transportability, thanks to linear tape file system (LTFS), and LTO-6 brings massive capacity for pennies a GB.

When we are dealing with petabytes of data, moving it all at once--across even the best of networks--is not an option. Protecting these large data sets as they are being created or modified, and using the right combination of disk and tape, is going to allow these archives to store all the information required to make them useful for years to come.

Follow Storage Switzerland on Twitter

George Crump is lead analyst of Storage Switzerland, an IT analyst firm focused on the storage and virtualization segments. Storage Switzerland's disclosure statement.

InformationWeek is conducting a survey to determine where enterprises stand on their IPv6 deployments, with a focus on security, training, budget, and readiness. Upon completion of our survey, you will be eligible to enter a drawing to receive a 16-GB Apple iPad. Take our D-Day for IPv6 Survey now. Survey ends May 11.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
4/28/2012 | 8:05:44 PM
re: How To Protect The Big Data Archive
Making Backups of your data is key!

How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of the Cloud Report
As the use of public cloud becomes a given, IT leaders must navigate the transition and advocate for management tools or architectures that allow them to realize the benefits they seek. Download this report to explore the issues and how to best leverage the cloud moving forward.
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on for the week of November 6, 2016. We'll be talking with the editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll