Google Wants Less Reliable Hard Disks

With less reliable hard disks tuned for collective operation, Google believes cloud data can be kept more affordably and securely.

Thomas Claburn, Editor at Large, Enterprise Mobility

February 25, 2016

3 Min Read
<p style="text-align:left">(Image: Google)</p>

iPhone Encryption: 5 Ways It's Changed Over Time

iPhone Encryption: 5 Ways It's Changed Over Time

iPhone Encryption: 5 Ways It's Changed Over Time (Click image for larger view and slideshow.)

Google wants to see storage technology evolve to meet the demands of cloud computing by adopting designs that are more affordable, more error prone, and better suited to collective operation.

In a research paper published Tuesday, Feb. 23, for the 2016 USENIX conference File and Storage Technologies (FAST 2016), Eric Brewer, VP of infrastructure at Google, calls for industry and academia to work together to adapt hard disk drives to current data center needs.

The problem, Brewer explains in a blog post, is that current "nearline enterprise" disks -- spinning hard disk drives as opposed to more expensive solid state drives -- are designed for traditional servers rather than large scale data centers supporting cloud computing. This is the fastest growing market, says Brewer, and it's likely to represent the majority of the market in the near future.

YouTube demonstrates why Google sees the need for change. YouTube users, according to Brewer, upload more than 400 hours of video every minute. With video requiring about 1 gigabyte of storage space per hour, that translates to more than a petabyte (1 million gigabytes) of new storage every day. And Brewer expects the rate of video ingestion to grow 10x every five years.

How then to accommodate our insatiable hunger for cat videos? Brewer argues that hard disks should be optimized to function as collections of disks rather than discrete devices associated with a single server. "This shift has a range of interesting consequences, including the counter-intuitive goal of having disks that are actually a little more likely to lose data, as we already have to have that data somewhere else anyway," Brewer says.

[ Read how Spotify rocks on Google's cloud. ]

Such hardware wouldn't actually lose data. It would be designed to facilitate the management disk errors at a higher level, through error correction and replication across multiple, coordinated disks. The result would be more affordable data center storage hardware. Individually, the disks would be more prone to errors. But collectively, they'd keep data secure while allowing improvements in capacity and performance.

The research paper, Disks for Data Centers, co-authored by Brewer, Lawrence Ying, Lawrence Greenfield, Robert Cypher, and Theodore T'so, also asserts that security must be improved as new use cases for storage are considered. It argues that security must be hardened to prevent unauthorized firmware changes and that encryption -- currently the cause of a major legal battle between Apple and the US government -- must be adapted to collections of disks through support for multiple keys. This would make it easier to secure data from different customers in shared disk space.

What have you done to advance the cause of Women in IT? Submit your entry now for InformationWeek's Women in IT Award. Full details and a submission form can be found here.

About the Author(s)

Thomas Claburn

Editor at Large, Enterprise Mobility

Thomas Claburn has been writing about business and technology since 1996, for publications such as New Architect, PC Computing, InformationWeek, Salon, Wired, and Ziff Davis Smart Business. Before that, he worked in film and television, having earned a not particularly useful master's degree in film production. He wrote the original treatment for 3DO's Killing Time, a short story that appeared in On Spec, and the screenplay for an independent film called The Hanged Man, which he would later direct. He's the author of a science fiction novel, Reflecting Fires, and a sadly neglected blog, Lot 49. His iPhone game, Blocfall, is available through the iTunes App Store. His wife is a talented jazz singer; he does not sing, which is for the best.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights