Think Today's Data Is Big? Wait 10 Years - InformationWeek
Data Management // Big Data Analytics
09:06 AM
Connect Directly

Think Today's Data Is Big? Wait 10 Years

If a petabyte won't qualify as big data, what will? Cleversafe's founder explains.

16 Top Big Data Analytics Platforms
16 Top Big Data Analytics Platforms
(Click image for larger view and slideshow.)

At what point does data make the leap to big data? There's no internationally sanctioned demarcation, of course, and it would appear that the rapid growth of unstructured data, particularly video, has the imaginary boundary on the move.

According to Chris Gladwin, founder and vice chairman of the object storage vendor Cleversafe, storage requirements have soared dramatically over the past decade, but that's just the start. Emerging technologies such as 4K video and cloud services will make today's big data storage needs seem laughably small by 2024.

Today, a petabyte of data -- 1,024 terabytes, to be exact -- probably meets many people's definition of "big data." The storage manufacturer Aberdeen, for example, sells a one-petabyte storage rack, aptly named the Petarack, for a cool $375,000.

Fast-forward 10 years, however, and a petabyte no longer will qualify as big -- at least not in the enterprise. "You think of it as a pretty big system today, but in 10 years, you won't even be able to buy systems that small," Gladwin told InformationWeek. "Today, a petabyte is half a cabinet, but you go out 10 years, and that's like part of a server. It won't even qualify as an enterprise-scale system."

[UK startup tackles the challenge of enabling computers to understand the spoken word. Read Natural Language Processing: Big Data's Role.]

Since Gladwin founded Cleversafe in 2004, the Chicago company has always focused on storage systems with very large capacity. A decade ago, only 60-70 companies in the world acquired a petabyte or more of new storage each year. "There was maybe one organization at that time that was at a 100-petabyte scale, and that was it. Now, 10 years later, the number of companies in the world that are deploying a petabyte or more of storage every year is around 7,000."

A decade from now, "when you look at the capacity-optimized segment of enterprise storage -- which is the big enterprise storage systems -- we're projecting that zero percent of the market will be systems that are a petabyte or less."

So if a petabyte won't qualify as big data, what will?

Perhaps an exabyte, or 1,024 petabytes. "Today's petabytes are tomorrow's exabytes. There are 5-10 organizations this year that will deploy at exabyte scale."

Currently, only a small group of companies need that much storage. "It's the Internet-deployed services that are storage-intensive. It can include social networking and companies that provide storage-as-a-service to others. The really big ones are at that scale."

The growing demand from cloud storage, particularly user-generated content, is driving this trend, as well. "You take a bunch of video files -- you get a hundred million customers making them -- and start storing them. The next thing you know, you're at exabyte-scale pretty quick." Government installations are reaching that level, too.

The arrival of 4K video -- and eventually 8K -- will also hasten the arrival of exabyte-level storage systems. "We're beginning the transition to 4K, and you can see the transition to 8K after that. Even with compression, each is basically a 4x jump" in storage. "If you really want to get to massive scale, you're not going to get there with numbers, text, and documents. You're really talking about big objects, and lots of them -- like videos or genomes, things like that."

Google is already at that scale, Gladwin said. "There's a billion people making videos, and they all land on YouTube."

Engage with Oracle president Mark Hurd, NFL CIO Michelle McKenna-Doyle, General Motors CIO Randy Mott, Box founder Aaron Levie, UPMC CIO Dan Drawbaugh, GE Power CIO Jim Fowler, and other leaders of the Digital Business movement at the InformationWeek Conference and Elite 100 Awards Ceremony, to be held in conjunction with Interop in Las Vegas, March 31 to April 1, 2014. See the full agenda here.

Jeff Bertolucci is a technology journalist in Los Angeles who writes mostly for Kiplinger's Personal Finance, The Saturday Evening Post, and InformationWeek. View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Chris Gladwin
Chris Gladwin,
User Rank: Apprentice
2/24/2014 | 2:21:51 PM
Re: How do the connections even keep up?
Certainly the growth of data will drive demand for increased network bandwidth. But innovative, new approaches - not just brute use of old methods that have worked in the past - will be required to handle this data growth from Terabyte to Petabyte to Exabyte scale.  For example, just replicating multiple copies of an Exabyte and sending those copies around a network to create location failure tolerance would create a huge wave of bandwidth demand and could overwhelm even an advanced network infrastructure.  Instead, new information coding techniques – such as Information Dispersal Algorithms – can be used to create multi-site failure tolerance while consuming less bandwidth than multiple copies.
Chris Gladwin
Chris Gladwin,
User Rank: Apprentice
2/24/2014 | 1:54:23 PM
Re: Beyond size
A huge factor driving data storage demands is the improving capability of smart machines, like cameras, scanners, genomics sequencers and more, to create ever-increasing amounts of digital information. All this additional information has a lot of benefits, but it also has a profound and increasing impact on the demand for data storage.
User Rank: Ninja
2/22/2014 | 11:50:40 AM
How do the connections even keep up?
Am I the only one who is thinking "ok, if we have that kind of data scale, how can you possibly keep up from a network perspective?"  8K video?  I am scared to barely watch .MKV files on my mobile device or via my home internet connection, what would the internet bills look like processing that size of data?  Can the network providers even keep up with the demand for processing and accessing such vast amounts of data?  That is the real question.  If you are an organization working with petabytes, or exabytes, the processing power to manage the information, outside of storage requirements, will be significant.  Throw in Cloud, and how you plan to access the information over the internet, and either you will be like Google and build your own networks, or we will have to see a revamp in the way data charges are set by the telecom providers.  

And don't even get me started on replication to cloud, or disaster recovery/business continuity.  How long would it take to work with that much data over today's networks?!
User Rank: Author
2/20/2014 | 11:19:21 PM
Re: Beyond size
Seeing 4K video now coming to the market, and knowing that's one small indication of the growing volume of data flooding into the world, it's easy to imagine how quaint TBs and PBs are already becoming.

Charlie Babcock
Charlie Babcock,
User Rank: Author
2/20/2014 | 9:00:51 PM
It's a good time to be a storage builder
Yes, big data is about more than size, but there's no denying the size of data sets is one of the few things keeping pace with Moore's Law. Fortunately, storage systems are almost keeping pace as well. But the projected sizes involved mean a lot of money is going to be spent on storage.
User Rank: Author
2/20/2014 | 11:22:01 AM
Beyond size
Big data is about more than size. We're hearing a lot that the variety of data types is at least as big a challenge ahead. What other "big" factors are causing headaches? 
Tech Vendors to Watch in 2019
Susan Fogarty, Editor in Chief,  11/13/2018
Getting DevOps Wrong: Top 5 Mistakes Organizations Make
Bill Kleyman, Writer/Blogger/Speaker,  11/2/2018
AI & Machine Learning: An Enterprise Guide
James M. Connolly, Executive Managing Editor, InformationWeekEditor in Chief,  9/27/2018
Register for InformationWeek Newsletters
Current Issue
The Next Generation of IT Support
The workforce is changing as businesses become global and technology erodes geographical and physical barriers.IT organizations are critical to enabling this transition and can utilize next-generation tools and strategies to provide world-class support regardless of location, platform or device
White Papers
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Sponsored Video
Flash Poll