Google Wants Less Reliable Hard Disks - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Cloud // Cloud Storage
News
2/25/2016
04:05 PM
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Google Wants Less Reliable Hard Disks

With less reliable hard disks tuned for collective operation, Google believes cloud data can be kept more affordably and securely.

iPhone Encryption: 5 Ways It's Changed Over Time
iPhone Encryption: 5 Ways It's Changed Over Time
(Click image for larger view and slideshow.)

Google wants to see storage technology evolve to meet the demands of cloud computing by adopting designs that are more affordable, more error prone, and better suited to collective operation.

In a research paper published Tuesday, Feb. 23, for the 2016 USENIX conference File and Storage Technologies (FAST 2016), Eric Brewer, VP of infrastructure at Google, calls for industry and academia to work together to adapt hard disk drives to current data center needs.

The problem, Brewer explains in a blog post, is that current "nearline enterprise" disks -- spinning hard disk drives as opposed to more expensive solid state drives -- are designed for traditional servers rather than large scale data centers supporting cloud computing. This is the fastest growing market, says Brewer, and it's likely to represent the majority of the market in the near future.

YouTube demonstrates why Google sees the need for change. YouTube users, according to Brewer, upload more than 400 hours of video every minute. With video requiring about 1 gigabyte of storage space per hour, that translates to more than a petabyte (1 million gigabytes) of new storage every day. And Brewer expects the rate of video ingestion to grow 10x every five years.

(Image: Google)

(Image: Google)

How then to accommodate our insatiable hunger for cat videos? Brewer argues that hard disks should be optimized to function as collections of disks rather than discrete devices associated with a single server. "This shift has a range of interesting consequences, including the counter-intuitive goal of having disks that are actually a little more likely to lose data, as we already have to have that data somewhere else anyway," Brewer says.

[ Read how Spotify rocks on Google's cloud. ]

Such hardware wouldn't actually lose data. It would be designed to facilitate the management disk errors at a higher level, through error correction and replication across multiple, coordinated disks. The result would be more affordable data center storage hardware. Individually, the disks would be more prone to errors. But collectively, they'd keep data secure while allowing improvements in capacity and performance.

The research paper, Disks for Data Centers, co-authored by Brewer, Lawrence Ying, Lawrence Greenfield, Robert Cypher, and Theodore T'so, also asserts that security must be improved as new use cases for storage are considered. It argues that security must be hardened to prevent unauthorized firmware changes and that encryption -- currently the cause of a major legal battle between Apple and the US government -- must be adapted to collections of disks through support for multiple keys. This would make it easier to secure data from different customers in shared disk space.

What have you done to advance the cause of Women in IT? Submit your entry now for InformationWeek's Women in IT Award. Full details and a submission form can be found here.

Thomas Claburn has been writing about business and technology since 1996, for publications such as New Architect, PC Computing, InformationWeek, Salon, Wired, and Ziff Davis Smart Business. Before that, he worked in film and television, having earned a not particularly useful ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
<<   <   Page 2 / 3   >   >>
SachinEE
50%
50%
SachinEE,
User Rank: Ninja
2/27/2016 | 1:46:27 AM
Re: Pretty intelligent
@Aroper: Like a cloud system within a cloud system. They are trying to fade the error correcting into the management system, just like a cloud, and together these disks represent the cloud. 
SachinEE
50%
50%
SachinEE,
User Rank: Ninja
2/27/2016 | 1:45:00 AM
Re: Pretty intelligent
@ASHU: I'm pretty sure these things have been standardized some years back. They are just dumbing down storage disks, nothing much. I think manufacturers won't have a problem dealing with this.
SachinEE
50%
50%
SachinEE,
User Rank: Ninja
2/27/2016 | 1:42:54 AM
Re: Pretty intelligent
@Yalanand: I think it only stores, one time write discs. I don't think they'll be able to use it under a space management system unless MOSFETs are included.
yalanand
50%
50%
yalanand,
User Rank: Ninja
2/27/2016 | 1:01:28 AM
Re: Pretty intelligent
@Gary_El: A shifting block data does have a security risk involved, if the data is being moved around in the same fashion again and again, hackers might successfully track it and hack into it. However I think Google has already thought about that in their implementation.
yalanand
50%
50%
yalanand,
User Rank: Ninja
2/27/2016 | 12:59:56 AM
Re: Pretty intelligent
@Whoopty: I recently heard about a glass disk that stores your data for billions of years. It is readable and can have multiple writes on it. I wonder when these will be implemented into big data sets.
Technocrati
50%
50%
Technocrati,
User Rank: Ninja
2/26/2016 | 7:21:50 PM
Re: Pretty intelligent

@Aroper-VEC       That is the first thing I thought of as well - A simple Raid.  We have to be careful to think just because Google mentions it - it has to be something innovative and new.  

There is nothing new about this concept, except Google appears to be pushing it.

Gary_EL
50%
50%
Gary_EL,
User Rank: Ninja
2/26/2016 | 3:41:10 PM
Re: Pretty intelligent
True, RAID is an established concept that has been around forever, but I wonder if data shifting back and forth between cheaper HDD's migh open up a security hole for someone to exploit? But, anyway you look at at it, HDD's are a "sunset technology". Open up a computer these days, and they the last remaining complex, moving mechanical part
Aroper-VEC
50%
50%
Aroper-VEC,
User Rank: Strategist
2/26/2016 | 11:23:30 AM
Re: Pretty intelligent
There is nothing new here. They're just adapting the premise behind distributed computing and applying it large scale for "cloud" storage. This is no different than RAID - which, literally, stands for a Redundant Array of INEXPENSIVE Disks. The problem with hard drives is growing capacity in scale and meeting IO demands. SSDs are no less error prone, or even more so in some instances, so they just want A LOT of hard drives to better balance the load but a lot of high-end hard drives would be very costly. They want to drive down the cost by shifting the error correcting focus away from the individual hard drives into management software. Though, this also isn't that dissimilar an approach to synapse computing in using a multitude of lesser expensive CPUs to distribute the workload with error correcting software managing the distribution and accounting for failed nodes.
Ashu001
100%
0%
Ashu001,
User Rank: Ninja
2/26/2016 | 9:50:56 AM
Re: Pretty intelligent
Whoopty,

There is a lot of stuff here which is still very much open to conjecture at this point of time.

I for one would prefer to wait and watch how HDD manufacturers react to this Paper and what changes they make going ahead in their Design & Infrastructure.

Its not easy for these guys to change everything on the whims and fancies of Software Companies-Today Google wants something,Tommorow MSFT wants something else,then Salesforce???

 
Whoopty
50%
50%
Whoopty,
User Rank: Ninja
2/26/2016 | 7:53:53 AM
Re: Pretty intelligent
I wonder if hard drives are really the solution for long term storage at this point though. NAND has come down in price so much in recent years that we may be looking at multiple terabyte SSDs in a few years rather than HDDs as storage. 
<<   <   Page 2 / 3   >   >>
News
How COVID is Changing Technology Futures
Jessica Davis, Senior Editor, Enterprise Apps,  7/23/2020
Slideshows
10 Ways AI Is Transforming Enterprise Software
Cynthia Harvey, Freelance Journalist, InformationWeek,  7/13/2020
Commentary
IT Career Paths You May Not Have Considered
Lisa Morgan, Freelance Writer,  6/30/2020
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Special Report: Why Performance Testing is Crucial Today
This special report will help enterprises determine what they should expect from performance testing solutions and how to put them to work most efficiently. Get it today!
Slideshows
Flash Poll