The Great File Migration Is Underway - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Cloud // Cloud Storage
Commentary
5/21/2015
08:06 AM
Connect Directly
Twitter
RSS
50%
50%

The Great File Migration Is Underway

As the amount of data in files grows, more and more will find their way in a great migration to cloud storage.

IT Hiring, Budgets In 2015: 7 Telling Stats
IT Hiring, Budgets In 2015: 7 Telling Stats
(Click image for larger view and slideshow.)

Two powerful trends are conspiring to force files out of the data center. Relentless data growth combined with the desire for high-performance flash arrays is making IT organizations rethink their file storage strategies. Files are being driven out of the data center, and they are heading to the cloud. As a result, the data center of today is faster, more efficient, and thinner than ever.

Flash storage arrays provide an ideal foundation for the IO-hungry workloads of virtual machine hypervisors, but they don’t solve the problem of files, which is one of scale and cost. A typical organization experiences 30% or more year-over-year compound growth in their file data. The problem is particularly acute in file-heavy industries such as design, engineering, media, and medical.

Todd Thomas, CIO at the Austin Radiological Association, illustrates this point when he shows their projected file growth. ARA is one of the largest medical imaging centers in the country, performing over a million exams annually. The chart below shows the file footprint just for mammography imaging alone. Today mammograms account for about 7 terabytes of storage, but that footprint is expected to double by the end of the year. Not only are there more files, but the files are getting much larger as imaging moves from 2D to 3D.

Although flash is becoming more affordable every year, it is no match for the rapid expansion in the file footprint. The widespread adoption of server virtualization drove the proliferation of SAN. This has been quickly followed by a demand for higher and higher performance from SAN. The data center wants to be all virtual. Virtualization improves the efficiency of the data center by making workloads more fluid. Electrical power can be directed to exactly where it is needed instead of having to over-spec hardware, only to find out later that even bigger hardware is required. As more server-class workloads break away from the underlying hardware through virtualization, disk IO becomes the bottleneck preventing hypervisors from handling heavier workloads.

[Want to learn more about how files are central to future storage systems? See NAS Is Sexy Again.]

Pain in the IO path has caused a slew of flash vendors to appear out of nowhere, which in just a few years have conquered the once unassailable mission-critical application workloads belonging to the Goliaths of the storage industry. Vendors like Pure Storage, SolidFire, Tegile, and Nimble Storage have pioneered the use of data de-duplication in flash in order to make the technology more affordable. At last, the all-virtual data center seems within reach. The last obstacle is files.

A successful transition to an all-virtual data center is predicated on being able to afford the premium that flash storage commands. Enlightened IT organizations are taking a two-pronged approach that consists of moving all of their server workloads to flash while pushing files to cloud-based NAS. The cloud has become such an appealing option, because it is essentially a giant data replication machine operated at a global scale by the likes of Amazon, Apple, Google, and Microsoft. As a result, the cloud makes NAS bottom-less, backup-less and much more affordable than anything built to run exclusively inside the data center.

This new way to store files uses synchronization to periodically update a master copy of not only every file, but also of the file version in these massive stable backends. Cloud NAS combines the stability and scale of the cloud into an on-premises appliance that fits neatly into the expectations for performance that exist in the modern data center.

This two-pronged approach to storage is no different from what has been happening to consumer laptops. Almost every new laptop today worth having ships with flash storage. Flash is faster, more reliable and less power hungry than a spinning hard drive, but it is also smaller and more expensive. But that’s OK, because, today there are also fewer files in a laptop.

Think of all that music being streamed instead of being held hostage inside your laptop. Apple recently introduced Photos, a whole new application for storing pictures. The exciting new feature: your pictures are actually stored in the Apple cloud, not your laptop. By removing photos from laptops, Apple hopes to sell faster, thinner, and more affordable laptops. Apple will also be making it easier and less risky for anyone to dump that old laptop because there is no real data in it. The master photo library stays in the cloud.

The great file migration is underway. The combined effect of continued file data expansion with a shift towards high performance arrays has caused organizations to adopt cloud NAS as a way to handle their file capacity needs. Cloud NAS has the additional benefit of storing the files outside the data center, free from the limitations of traditional storage. This approach not only simplifies hardware refreshes, but also ensures that the files are safe no matter what happens at the data center.

The problem with files is one of scale. The cloud is practically infinite. The great file migration to the cloud is inevitable.

[Did you miss any of the InformationWeek Conference in Las Vegas last month? Don't worry: We have you covered. Check out what our speakers had to say and see tweets from the show. Let's keep the conversation going.]

Andres Rodriguez is CEO of Nasuni, a supplier of enterprise storage using on-premises hardware and cloud services, accessed via an appliance. He previously co-founded Archivas, a company that developed an enterprise-class cloud storage system and was acquired by Hitachi Data ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
5/22/2015 | 6:43:19 PM
A global file systemj with headquarters in the cloud
In his Sept. 25, Andres talked about the need for a single global file system based outside the enterprise data center. It would  make use of the cloud's characteristic of keeping three copies of the data in order to be able to survive a hardware failure. If one or more copies is kept geographically separate as well as in a separate availability zone, cloud storage lends itself to strong data survivability characteristics through this cloud practice, without layering on top its own "make a copy: replications. In corporated cloud architecture as a given and available for global file system creation, accessible by all. I think we're headed i that direction.
Gary_EL
50%
50%
Gary_EL,
User Rank: Ninja
5/22/2015 | 1:00:27 AM
Re: What makes file storage in the cloud inevitable?
I don't believe that choices will be so stark. The files you use most often will stay stored onsite, because it still takes some time (and money) to constantly download and upload them. But, they still need to be periodically (OFTEN!)  backed up to a safe place offsite - on the cloud. But, why take up expensive space at your business, when seldom used files can be stored where the rent is cheap, in a remote data site?

But, that's today's paradigm. Watch out for a new memory technology called memristors that Hewlett Packard is betting the farm on. It has the potential to change things around.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
5/21/2015 | 8:46:47 PM
What makes file storage in the cloud inevitable?
Is Andres right? Will files inevitably find their way into some form of cloud storage? Or will the capacities of on-premises storage systems keep up with the wild growth of information? And if you have an on-premises solution, will it have the automated replication and second site storage of the cloud?
Slideshows
DeveloperWeek NY and MongoDB World Tackle Transformation
Joao-Pierre S. Ruth, Senior Writer,  6/25/2019
Commentary
GDPR One Year Later: Was the Hype Worth It?
Guest Commentary, Guest Commentary,  6/19/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
A New World of IT Management in 2019
This IT Trend Report highlights how several years of developments in technology and business strategies have led to a subsequent wave of changes in the role of an IT organization, how CIOs and other IT leaders approach management, in addition to the jobs of many IT professionals up and down the org chart.
Slideshows
Flash Poll