The Great File Migration Is Underway

As the amount of data in files grows, more and more will find their way in a great migration to cloud storage.

Andres Rodriguez, CEO, Nasuni

May 21, 2015

5 Min Read

IT Hiring, Budgets In 2015: 7 Telling Stats

IT Hiring, Budgets In 2015: 7 Telling Stats

IT Hiring, Budgets In 2015: 7 Telling Stats (Click image for larger view and slideshow.)

Two powerful trends are conspiring to force files out of the data center. Relentless data growth combined with the desire for high-performance flash arrays is making IT organizations rethink their file storage strategies. Files are being driven out of the data center, and they are heading to the cloud. As a result, the data center of today is faster, more efficient, and thinner than ever.

Flash storage arrays provide an ideal foundation for the IO-hungry workloads of virtual machine hypervisors, but they don’t solve the problem of files, which is one of scale and cost. A typical organization experiences 30% or more year-over-year compound growth in their file data. The problem is particularly acute in file-heavy industries such as design, engineering, media, and medical.

Todd Thomas, CIO at the Austin Radiological Association, illustrates this point when he shows their projected file growth. ARA is one of the largest medical imaging centers in the country, performing over a million exams annually. The chart below shows the file footprint just for mammography imaging alone. Today mammograms account for about 7 terabytes of storage, but that footprint is expected to double by the end of the year. Not only are there more files, but the files are getting much larger as imaging moves from 2D to 3D.

Although flash is becoming more affordable every year, it is no match for the rapid expansion in the file footprint. The widespread adoption of server virtualization drove the proliferation of SAN. This has been quickly followed by a demand for higher and higher performance from SAN. The data center wants to be all virtual. Virtualization improves the efficiency of the data center by making workloads more fluid. Electrical power can be directed to exactly where it is needed instead of having to over-spec hardware, only to find out later that even bigger hardware is required. As more server-class workloads break away from the underlying hardware through virtualization, disk IO becomes the bottleneck preventing hypervisors from handling heavier workloads.

[Want to learn more about how files are central to future storage systems? See NAS Is Sexy Again.]

Pain in the IO path has caused a slew of flash vendors to appear out of nowhere, which in just a few years have conquered the once unassailable mission-critical application workloads belonging to the Goliaths of the storage industry. Vendors like Pure Storage, SolidFire, Tegile, and Nimble Storage have pioneered the use of data de-duplication in flash in order to make the technology more affordable. At last, the all-virtual data center seems within reach. The last obstacle is files.

A successful transition to an all-virtual data center is predicated on being able to afford the premium that flash storage commands. Enlightened IT organizations are taking a two-pronged approach that consists of moving all of their server workloads to flash while pushing files to cloud-based NAS. The cloud has become such an appealing option, because it is essentially a giant data replication machine operated at a global scale by the likes of Amazon, Apple, Google, and Microsoft. As a result, the cloud makes NAS bottom-less, backup-less and much more affordable than anything built to run exclusively inside the data center.

This new way to store files uses synchronization to periodically update a master copy of not only every file, but also of the file version in these massive stable backends. Cloud NAS combines the stability and scale of the cloud into an on-premises appliance that fits neatly into the expectations for performance that exist in the modern data center.

This two-pronged approach to storage is no different from what has been happening to consumer laptops. Almost every new laptop today worth having ships with flash storage. Flash is faster, more reliable and less power hungry than a spinning hard drive, but it is also smaller and more expensive. But that’s OK, because, today there are also fewer files in a laptop.

Think of all that music being streamed instead of being held hostage inside your laptop. Apple recently introduced Photos, a whole new application for storing pictures. The exciting new feature: your pictures are actually stored in the Apple cloud, not your laptop. By removing photos from laptops, Apple hopes to sell faster, thinner, and more affordable laptops. Apple will also be making it easier and less risky for anyone to dump that old laptop because there is no real data in it. The master photo library stays in the cloud.

The great file migration is underway. The combined effect of continued file data expansion with a shift towards high performance arrays has caused organizations to adopt cloud NAS as a way to handle their file capacity needs. Cloud NAS has the additional benefit of storing the files outside the data center, free from the limitations of traditional storage. This approach not only simplifies hardware refreshes, but also ensures that the files are safe no matter what happens at the data center.

The problem with files is one of scale. The cloud is practically infinite. The great file migration to the cloud is inevitable.

[Did you miss any of the InformationWeek Conference in Las Vegas last month? Don't worry: We have you covered. Check out what our speakers had to say and see tweets from the show. Let's keep the conversation going.]

About the Author(s)

Andres Rodriguez

CEO, Nasuni

Andres Rodriguez is CEO of Nasuni, a supplier of enterprise storage using on-premises hardware and cloud services, accessed via an appliance. He previously co-founded Archivas, a company that developed an enterprise-class cloud storage system and was acquired by Hitachi Data Systems in 2006. After the acquisition, he served as Hitachi's CTO of Files Services. Prior to founding Archivas, Rodriguez was CTO of The New York Times. He received a Bachelor of Science in engineering and a Master's in physics from Boston University. He holds several patents for system designs.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights