Cloud // Cloud Storage
Commentary
4/27/2012
10:49 AM
George Crump
George Crump
Commentary
Connect Directly
RSS
E-Mail
50%
50%
Repost This

How To Protect The Big Data Archive

Your big data disaster recovery strategy must consider size, cost, and the unpredictable, but inevitable, need to access old data.

Protecting a big data archive is different than protecting big data analytics--or it should be. While both types of big data are, well big, big data archives typically have the larger capacity. Full backups are not an option (too much data), and a disk-only backup strategy may not be realistic (too expensive). Big data archives must have data protection embedded into the architecture, as the data sets are too large to run a separate process.

As we discussed in a recent column big data archive environments are storing dozens of petabytes of content, often video, audio, or images. That content can sit idle for years and then a portion of it becomes active, triggered by some event, for a period of time. A common example: When celebrities die or get into trouble, video and photos from their past (that were infrequently accessed) become heavily requested. The size of these environments makes backup almost impossible, but the likelihood of accessing that data makes protection a top requirement.

The challenge presented by the data set's size and access pattern is such that creating a disk-only archive store may not be practical. The challenge is not one of capacity, as we discuss in our article "Is Your File Server Choking." Object-based storage from cloud infrastructure suppliers has eliminated the key scaling issues facing traditional file systems, adding near-infinite capacity and dealing with trillions of files. The challenge is the cost associated with providing a near-infinite capacity, disk-based storage system and backing it up via replication to another disk-based, near-infinite capacity system.

As a result, tape has a role, and potentially a prominent one, in big data archives. Archive file systems, like those offered by members of the Active Archive Alliance, have the capability to support tape as an integral part of the environment. What this means is that a large percentage of the active data sets can still be kept on disk for instant access, but can also be copied to a tape or two upon creation. The second tape can be used for backup and disaster recovery. This means "backup" happens as the data is created or modified, not as a single, separate, nightly process. As the disaster-recovery tape fills, it can be moved to a secure offsite location. It also means that tape can be the only location of that data, eventually scrubbing the data from disk.

This is not an "abandon disk" recommendation; it is a recommendation to be realistic. We suggest that you put as much data on disk as makes sense and can be afforded. Disk is needed for the data that will become hot, driven by the next world event. It is also needed for recently created data, since that is the most likely to be accessed.

Tape is needed to back up all the information and make sure the petabytes of old information that won't be accessed is cost-effectively stored. As we discuss in our article "Comparing LTO-6 to Scale Out Storage for Long-Term Retention," linear tape open (LTO) now has transportability, thanks to linear tape file system (LTFS), and LTO-6 brings massive capacity for pennies a GB.

When we are dealing with petabytes of data, moving it all at once--across even the best of networks--is not an option. Protecting these large data sets as they are being created or modified, and using the right combination of disk and tape, is going to allow these archives to store all the information required to make them useful for years to come.

Follow Storage Switzerland on Twitter

George Crump is lead analyst of Storage Switzerland, an IT analyst firm focused on the storage and virtualization segments. Storage Switzerland's disclosure statement.

InformationWeek is conducting a survey to determine where enterprises stand on their IPv6 deployments, with a focus on security, training, budget, and readiness. Upon completion of our survey, you will be eligible to enter a drawing to receive a 16-GB Apple iPad. Take our D-Day for IPv6 Survey now. Survey ends May 11.

Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
OmegaX
50%
50%
OmegaX,
User Rank: Apprentice
4/28/2012 | 8:05:44 PM
re: How To Protect The Big Data Archive
Making Backups of your data is key!

http://harddrivefailed.blogspo...
Google in the Enterprise Survey
Google in the Enterprise Survey
There's no doubt Google has made headway into businesses: Just 28 percent discourage or ban use of its productivity ­products, and 69 percent cite Google Apps' good or excellent ­mobility. But progress could still stall: 59 percent of nonusers ­distrust the security of Google's cloud. Its data privacy is an open question, and 37 percent worry about integration.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Video
Slideshows
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.