What Netflix Could Do For Cloud Computing

[ Editor's note: InformationWeek started a conversation in the cloud community with two recent opinion columns that take opposing views on Netflix's Cloud Prize, a $100,000 contest designed to advance cloud computing in 10 categories. Netflix has committed itself to using Amazon Web Services instead of building out its own data centers, and has shifted all but its core systems into Amazon EC2. So it's heavily committed to the architecture and continued development of the AWS cloud, as opposed to more open alternatives. Rackspace CTO John Engates is a leading spokesman for OpenStack, the primary open source alternative to both VMware as a private cloud architecture and Amazon as a public cloud architecture. Here, Engates weighs into the debate on Netflix and cloud innovation. Share your opinion in the comments section at the end of this column. ]

It's popular to kick Netflix for its outages, for building on a proprietary cloud service or for not streaming enough of its extensive movie library. The company is often depicted, including in a recent InformationWeek column, as a poster child for cloud lock-in -- the prisoner of Amazon.

There's some validity to each of these criticisms. But I think they miss a larger point. IT is rapidly evolving beyond the Amazon-focused "Cloud v1.0," whose shortcomings Joe Masters Emison recently called out in these pages. Thanks to the 165 companies who sponsor OpenStack, and the 860 developers who have contributed to that open-source project, we are well on our way to open standards that will end customer reliance on any one cloud provider's proprietary code. Netflix isn't going to stop the open cloud movement. But it's a shame that it doesn't join it, because it's in a unique position to take a leading role -- for its own benefit, and that of all cloud users.

The truth is that Netflix has been on the cutting edge of cloud innovation right from the beginning. It has some of the planet's smartest and most creative developers and systems architects working on a really big problem. And they're coming up with amazing solutions. The team there is probably at the pinnacle of engineering prowess when it comes to building a complex business at scale on a public cloud.

[ Read the two opposing views that started a debate on this topic: How Netflix Is Ruining Cloud Computing and Netflix's Cloud Contest: More Companies Should Follow Suit. ]

You don't have to take my word for it. Netflix has open-sourced many of the tools it created and you can see for yourself how hard they've worked to make a workable service on a closed platform. It's a gift to the community of cloud users and something that can help accelerate cloud adoption. I'm excited to see the results of its recently announced developer challenge.

But it could be doing much more. It could be making all of public cloud computing stronger, more stable and better able to perform at scale. Instead, it has limited the scope of its work to just adding tools to make a single, proprietary cloud run a little less erratically.

So I take exception when I read that "Netflix Is Ruining Cloud Computing" by creating tools to help people better use a major cloud service. It's not ruining cloud; it's just not contributing at the elemental level. It's adding intricate workarounds instead of rolling up its sleeves and fixing the underlying problems.

A perfect example of this is the Simian Army. This is a set of tools that help developers wrap their heads around the idea that their instances could fail at any time. When you can't see what's driving your infrastructure, you just have to accept regular instance failure. It's just a fact of life for some public cloud users (including many smaller ones who lack Netflix-scale resources to architect apps across multiple "availability zones" and geographies.)

Of course cloud outages don't have to be something to engineer around. Power users such as Netflix could actually work directly on the code that runs the cloud and make it more stable and scalable for themselves and other users. That's the beauty of OpenStack. Companies and individuals can improve the underlying architecture of cloud computing instead of laboring to create the ever-more complex workarounds and band-aids that are required to use a closed cloud.

That's what bums me out about the Netflix challenge. I'd love to see that creative energy applied to actually making better cloud code. Anyone can find the weaknesses in an open-source cloud operating system and improve on them. Just fix it. Then everyone who uses an open cloud can benefit from those insights, innovations and improvements.

Netflix isn't ruining cloud computing. It's pushing the boundaries of what's possible in the closed system it's committed itself to. But I'd like to see it doing more. Creating open-source tools on a closed cloud isn't enough.

We at Rackspace believe an open cloud takes us all further. That's why we open-sourced the cloud by co-founding OpenStack, in collaboration with NASA. That's why we contributed millions of dollars worth of our code to the project, then turned OpenStack over to an independent foundation. We're delighted that OpenStack now has more than 8,200 registered members in more than 100 countries, and big corporate sponsors such as Dell, HP, Red Hat and IBM (which recently announced that all of its new cloud hardware and software will be based on OpenStack.)

The cloud is still a new technology. It needs to be improved at the elemental level, not bettered by a series of bolt-ons. The open cloud will be made more reliable by the wisdom of crowds -- by the sheer number of developers, from many companies and cultures, working to make it better each day. It's a pity that Netflix has isolated itself. But the rest of us will work together to take this thing as far as we can.