Cloud // Platform as a Service
Commentary
3/25/2013
02:00 PM
Connect Directly
LinkedIn
Google+
Twitter
RSS
E-Mail
50%
50%

How Netflix Is Ruining Cloud Computing

A laser focus on Amazon Web Services and seeming disregard for next-gen best practices could spell lock-in, and derail real IaaS competition.

On March 13, Netflix announced $100,000 in prize money for the developers who do the most to improve its open source tools for controlling and managing application deployments on cloud computing. Before spearheading this contest, Netflix's cloud architect, Adrian Cockcroft, released many internal Netflix tools as open source. Currently, 8 cloud-architecture-specific tools are available from Netflix, and Cockcroft has been very open in sharing his and Netflix's knowledge in public forms.

In theory, all of this should be wonderful. In reality, however, it's likely to leave cloud computing with an enormous hangover of subpar practices and architectures for years to come. Netflix is the poster child for "Cloud Computing v1.0" and demonstrates both the enormous benefits and troubling problems. Cloud Computing v1.0 is a strictly an Amazon Web Services affair -- it was first, and no other provider had the core features necessary to build comparable applications (think multiple availability zones and EBS with snapshots and quick restores). So it makes sense that Netflix embraced AWS; it saw huge benefits in being able to deploy and scale its service using the interfaces and architectures that were possible when AWS launched.

But Netflix has also suffered repeatedly at the hands of Cloud Computing v1.0 with four outages in 2012 alone, which certainly points to the possibility for some improvement in the high availability of its service. Of note, the Christmas Eve outage is perhaps most troubling from a "v1.0" perspective, as it was solely the result of Netflix's reliance on a less-necessary AWS service for load balancing, which could have been handled in any number of other ways to increase server availability.

[ Check out our new InformationWeek cloud computing comparison of 13 top PaaS vendors: Cloud Computing Comparison: PaaS Providers. ]

The reason the Netflix contest is likely to leave organizations worse off is because it thoroughly embraces this "Cloud Computing v1.0" mindset, both from an "AWS-is-the-only-vendor" standpoint as well as from an architectural standpoint. While it's arguable that there still isn't (quite yet) another infrastructure-as-a-service (IaaS) vendor that has a thoroughly tested core feature set, unless you just walked out of the tattoo parlor with "#AWS" on your shoulder, you know it won't be long. And all companies running on AWS should be looking forward to the rise of additional IaaS vendors, like those in our IaaS buyer's guide, for two reasons: higher availability and price competition.

Every cloud architect should know that it's only a matter of time before organizations have applications deployed across the world on many different IaaS providers in many different data centers, based on request volume and location in combination with a market for computing resources that changes price constantly. Locking yourself down to AWS today, for greenfield cloud architectures, would be the equivalent of deciding to develop an iPhone-only application when you know you'll have to support iPads, Android and others in the future.

In addition to the annoying AWS-centrism of the Netflix contest, there's a deeper problem: Some of Netflix's tools embrace a cloud architecture that was fine in the days of Cloud Computing v1.0 but that will look increasingly suspect as time goes on. I know that it's hard to throw out code and systems that are working fine, especially when they still look pretty good -- and often, squeaking out a bit more time is the right internal decision for an individual company. But instead of just wringing out the last bits of value, Netflix is throwing significant money at the rest of the world, asking everyone to embrace and extend their tools and code that are not particularly good practices for future cloud architectures.

Perhaps the best example of a bad-practice Netflix tool is Aminator. Aminator helps you build Amazon Machine Images (AMIs) easily, based on a "base" AMI and a package of code. "I must have produced about 25,000 Ubuntu AMIs," raved one excited early user. There's just one problem: It's hard to understand when this would ever be a good idea. Several years ago, spawning tons of images would have been a somewhat acceptable way to roll out a revised version of an application (due to application code, operating system, and/or server software). But today we have widespread adoption of configuration management tools like Chef and Puppet that make massive AMI creation a subpar practice at best. Amazon Web Services itself recently rolled out a service called OpsWorks, which would be a significantly better way to handle deploying applications -- it uses Chef.

There are other less-bad tools, but many bear the mark of having to architect around a number of issues that have since been more or less resolved; it's a bit like an open source project that relies heavily on SOAP instead of being RESTful. For example, Edda, which figures out what cloud resources you're using at AWS, just seems like something that had to be built because no one properly set up how resources should be requested and deployed. And Asgard, a very cool tool from 2010 for managing a variety of different applications across AWS, would be a hard sell as a best-of-breed tool today compared with other open source options, notably Scalr and Chef.

This is not to say that all of Netflix's open source cloud tools fit into this mold. Denominator is a great DNS manager (because it's multi-cloud), and Simian Army is a fabulous, ground-breaking idea for testing cloud architectures (it is, unfortunately, AWS-only).

There's a possibility that the Netflix contest will help lead the world toward Cloud Computing v2.0 and beyond by embracing multi-cloud architectures that use orchestration and configuration management in optimal ways. However, I am skeptical on both fronts. Cockcroft's public comments suggest little interest in using other cloud vendors. A good chunk of the prize money is in AWS credits, and Amazon's CTO is a judge; all this points to a very AWS-centric mindset. Moreover, the fact that Netflix just released Aminator last week indicates to me that Netflix is happy to roll out whatever tools they've built, regardless of whether they fit in with a best-practices modern cloud architecture.

But please, Netflix, prove me wrong. Embrace a less proprietary, more highly available, more standardized cloud -- and put Google's Urs Hölzle on the panel while you're at it. #UrsForNetflixJudge

Cloud Connect returns to Silicon Valley, April 2-5, 2013, for four days of lectures, panels, tutorials and roundtable discussions on a comprehensive selection of cloud topics taught by leading industry experts. Join us in Silicon Valley to see new products, keep up-to-date on industry trends and create and strengthen professional relationships. Use Priority Code MPIWK by March 30 to save an extra $200 off the advance price of Conference Passes. Register for Cloud Connect now.

Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
Deirdre Blake
50%
50%
Deirdre Blake,
User Rank: Apprentice
3/26/2013 | 3:23:03 PM
re: How Netflix Is Ruining Cloud Computing
So the point is....? Netflix shouldn't hold any developer competitions until a "real" contender to AWS emerges?
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/26/2013 | 7:06:34 PM
re: How Netflix Is Ruining Cloud Computing
The point is that Netflix is holding a developer competition which--as it is designed--will likely produce tools and encourage practices more consistent with "clouding computing v1.0". It's fairly clear how tools should be built to work with multiple clouds--where you should build abstraction layers, and how you should embrace more open standards and choices instead of choosing proprietary ones. Most of Netflix's tools don't do that today (and in particular, Asgard's place at the center of things and reliance upon so many proprietary AWS services and API calls), and how the tools are currently built and how the contest is designed are all pointing off in a direction that goes away from a multi-cloud, standardized-configuration-management design.

Look, if this were Microsoft hosting a contest to improve its cloud tools, what I wrote would be a non-story--of course we should all be skeptical about whether what Microsoft is offering would be best-of-breed and useful. The problem is that very few people in the cloud world (at least those to whom I've spoken) seem to be viewing this Netflix/AWS contest (let's not leave AWS out, since it's very, very clearly a joint contest, with Werner and the AWS prizes) with a similar skeptical eye.
treehousetim
50%
50%
treehousetim,
User Rank: Apprentice
3/26/2013 | 4:26:39 PM
re: How Netflix Is Ruining Cloud Computing
OpenStack offers flexibility and choice through a highly engaged community of over 6,000 individuals and over 190 companies including Rackspace-«, Dell, HP, IBM, and Red Hat-«.

http://www.rackspace.com/cloud...
tw426
50%
50%
tw426,
User Rank: Apprentice
3/26/2013 | 5:09:28 PM
re: How Netflix Is Ruining Cloud Computing
Really? So just because AWS doesn't meet your classification of a modern, standard way of doing things, you abandon it? Replacing a working system for those reasons is a funny mindset... and one of the benefits of opening a 'contest' is to look at new ideas with an open attitude. Who is to say that something even better won't come out of this experiment?

It sounds to me like you may just want to bash NetFlix (as if they need any help from you) - they have demonstrated that they are perfectly able to do that themselves!
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/26/2013 | 7:13:09 PM
re: How Netflix Is Ruining Cloud Computing
Wow--nothing in my piece says anything about AWS not being modern or me abandoning it. AWS still gets the vast majority of my cloud spend today, and I think it's a wonderful service.

The point is that a *cloud architecture* that *only works with one cloud* is not a good cloud architecture going forward. Good cloud architectures will work with multiple clouds. If I am hiring you as a cloud architect today, and you tell me that you are going to create a greenfield cloud architecture that is going to hook directly into AWS APIs and that you are fundamentally uninterested in testing or supporting any other clouds in the future, then I would fire you on the spot.

I am also a big fan of Netflix's services, of their corporate culture, and of the fact that they're willing to be so open about so many things, including releasing so much source code. (I think I've been a Netflix member for more than 10 years now). The problem is that I am concerned about the vast number of people and organizations who are going to take Netflix's cloud deployment as a reference architecture, which will help Netflix, but will fundamentally set cloud architecture practices back to 2010.

All that said, if the result of this contest is to bring Netflix's tools into the future and make them multi-cloud, and use standardized configuration management, then that will be excellent (as I say at the end of my piece). However, for the many reasons I outlined in my piece, I am deeply skeptical of whether that will happen.
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
3/26/2013 | 5:13:05 PM
re: How Netflix Is Ruining Cloud Computing
Sounds more like the point is big users of capacity should promote a cloud free market in which there's a real choice. Amazon has been great about lowering prices, but perhaps it would lower prices even more if it had real competition. Trouble is, it's tough for anybody except those operating at Google or, perhaps, Microsoft scale to achieve or even approach the economies of scale already established by Amazon.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/26/2013 | 7:16:43 PM
re: How Netflix Is Ruining Cloud Computing
Yes, I absolutely agree with this. Big users of capacity need to have architectures that can take advantage of multiple IaaS vendors (and I agree that we're only really looking at Google and Microsoft, at least today), and need to not look like Netflix's architecture, which requires so many proprietary AWS services (e.g., DynamoDB) as it stands today.
adrianco
50%
50%
adrianco,
User Rank: Apprentice
3/27/2013 | 12:41:59 AM
re: How Netflix Is Ruining Cloud Computing
Correction: The only DynamoDB support in NetflixOSS is contributed code by someone who was not working at Netflix at the time (a former employee). Netflix mostly uses Cassandra.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/27/2013 | 8:40:41 PM
re: How Netflix Is Ruining Cloud Computing
Sorry--replace DynamoDB with SimpleDB. Same point: proprietary to AWS.
lgarey@techweb.com
50%
50%
lgarey@techweb.com,
User Rank: Apprentice
3/26/2013 | 7:26:25 PM
re: How Netflix Is Ruining Cloud Computing
Agreed Doug. It's to NetFlix's own benefit to nurture a vibrant set of IaaS providers. Lorna Garey, IW Reports
adrianco
50%
50%
adrianco,
User Rank: Apprentice
3/26/2013 | 6:20:30 PM
re: How Netflix Is Ruining Cloud Computing
There should be a techblog.netflix.com post in the next day or so that will give more context to the Cloud Prize and clarify most of the points above. However I will address some of the specific issues here.

Cloud 1.0 vs. 2.0?
I would argue that the way most people are doing cloud today is to forklift part of their existing architecture into a cloud and run a hybrid setup. That's what I would call Cloud 1.0. What Netflix has done is show how to build much more agile green field native cloud applications, which might justify being called Cloud 2.0. The specific IaaS provider used underneath, and whether you do this with public or private clouds is irrelevant to the architectural constructs we've explained.

Outages
The outages that have been mentioned were regional, they didn't apply to Netflix operations in Europe for example. Our current work is to build tooling for multi-regional support on AWS (East cosat/West coast), including the DNS management that was mentioned. This removes the failure mode with the least effort and disruption to our existing operations.

Portability
Other cloud vendors have a feature set and scale comparable to AWS in 2008-2009. We're still waiting for them to catch up. There are many promises but nothing usable for Netflix itself. However there is demand to use NetflixOSS for other smaller and simpler applications, in both public and private clouds, and Eucalyptus have demonstrated Asgard, Edda and Chaos Monkey running, and will ship soon in Eucalyptus 3.3. There are signs of interest from people to add the missing features to OpenStack, CloudStack and Google Compute so that NetflixOSS can also run on them.

Edda
You've completely missed the point of Edda. It does three important things. 1) if you run at large scale your automation will overload the cloud API endpoint, Edda buffers this information and provides a query capability for efficient lookups. 2) Edda stores a history of your config, it's a CMDB that can be used to query for what changed. 3) Edda cross integrates multiple data sources, the cloud API, our own service registry Eureka, Appdynamics call flow information and can be extended to include other data sources.

AMInator
If you want to spin up 500 identical instances, having them each run Chef or Puppet after they boot creates a failure mode dependency on the Chef/Puppet service, wastes startup time, and if anything can go wrong with the install you end up with an inconsistent set of instances. By using AMInator to run Chef once at build time, there is less to go wrong at run time. It also makes red/black pushes and roll-backs trivial and reliable.

Cloud Prize
The prize includes a portability category. It's a broad category and might be won by someone who adds new language support to NetflixOSS (Erlang, Go, Ruby?) or someone who makes parts of NetflixOSS run on a broader range of IaaS options. The reality is that AWS is actually dominating cloud deployments today, so contributions that run on AWS will have the greatest utility by the largest number of people. The alternatives to AWS are being hyped by everyone else, and are showing some promise, but have some way to go.

We hope that NetflixOSS provides a useful driver for higher baseline functionality that more IaaS APIs can converge on, and move from 2008-era EC2 functionality to 2010-era EC2 functionality across more vendors. Meanwhile Netflix itself will be enjoying the benefits of 2013 AWS functionality like RedShift.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/26/2013 | 9:26:54 PM
re: How Netflix Is Ruining Cloud Computing
Adrian--

I read this as a "tree" response to a "forest" issue, but I'll respond with respect to both forest and trees.

The forest is this: Netflix's cloud architecture--as seen through public talks and open source code--is fundamentally (a) so intertwined with AWS as to be essentially inseparable, and (b) significantly behind the best *general* open options for configuration management and orchestration. It also is far from "the Unix way" of having encapsulated/abstracted tools that can be interchanged with others to build a best-in-breed architecture.

Your answer doesn't really do anything to do address this "forest" argument: you defend the complete reliance that Netflix and (most of) its tools have on AWS based upon an analytical database that is really beside the point as far as cloud architectures go. (Don't get me wrong--I think RedShift is *awesome*, but its presence is completely irrelevant to a generalized reference cloud architecture, which is the power of NetflixOSS that's so concerning).

Your defense of AMInator and Edda (I wish you'd defend Asgard also!) is ultimately a defense of why those solutions work for Netflix and its application and current architecture--but that's not the point. Obviously you're a smart and capable architect and you have reasons for using them at Netflix. The point is that--as they stand today--they're not promoting good *generalized* application architectures. You should be promoting Chef before you promote a tool that essentially encourages people make horrible design decisions (in lieu of using Chef at all). You should be defending Netflix tools based upon standardized, reference deployments, not based on launching 500 VMs of the same exact machine which *is not exactly a common use case for the cloud*.

Look--it's possible to write awesome and fabulous PHP code, but most PHP developers don't. One of the reasons why Netflix is now choosing Python is because the generalized Python developer writes consistent and good code. (We chose Python for the same reasons you did). But to someone who has no idea what a good cloud deployment looks like, the way AMInator sits out there--you're going to see a lot more people like the guy super-psyched to have built 25,000 AMIs over Twitter.

The overall point of the piece is this: Netflix has a lot of power and clout in the cloud architecture world, and there are a whole lot of people looking to Netflix for guidance on how to deploy on the cloud. Netflix has made some choices (the "forest" above) that are flat-out bad choices if you take anything like a long-term approach to your cloud architecture. There is no historical precedent that you can cite as being a good example for being so intertwined with a single IT vendor. And it's way more important for people deploying on the cloud to know and understand configuration management than it is for them to have a tool that--as far as its public users go--are using to bypass CM entirely.

adrianco
50%
50%
adrianco,
User Rank: Apprentice
3/26/2013 | 11:49:08 PM
re: How Netflix Is Ruining Cloud Computing
The Netflix cloud platform *is* the abstraction we built to isolate our streaming apps from dealing directly with AWS. The rest of it is the things we had to build to get things done. The structure of NetflixOSS is based on lots of separate services that can be combined in many ways or used individually just like the 'Unix way'.

You're advocating a lowest common denominator for cloud architecture, I'm pointing out that this roots you in 2008, and ignores everything that AWS has developed in response to the needs of their customers in the following five years. To effectively use tools like Asgard cloud APIs need to implement ideas from 2010, and it's obviously possible, since Eucalyptus have done just that, and other cloud API vendors are looking at it.

We really do push code many times every day using tens or hundreds of instances of a baked AMI. We have several hundred different services running. If you don't want to run at scale, you don't need to do this, but if scale and high availability and fast startup matters, AMInator solves the problem. The developer you mentioned having built 25,000 AMIs is our engineer who has been building and testing AMInator for the last few months.

There are many historical precedents for using a single IT vendor. You pick the ones that work at scale. There's also a philosophical difference between the concerns of Operations people who want multiple vendors to save money, and Developers who want a single vendor to get things done more quickly. For example developers don't write SQL code that transparently works on Oracle, SQLserver and MySQL they picked one of them to start with and port if they have to.

You mention Zynga, which I regard as a case study of why it's a bad idea to spend hundreds of millions of dollars building datacenters just before your business model collapses. Most of the people who built Z-cloud are gone now.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/27/2013 | 10:06:50 AM
re: How Netflix Is Ruining Cloud Computing
Again, the issue is not about whether Netflix's current business decisions work for Netflix; the issue is about whether Netflix's tools and contest are beneficial for the enterprise that is considering how to move to the cloud. Running "at scale" for Netflix is thoroughly unlike running "at scale" for the vast majority of enterprise cloud need.

I'm not advocating a "lowest common denominator" as much as I'm advocating a fundamental set of best practices that one should master before getting into questions like, "how to launch 5,000 VMs in six continents within 10 minutes." If someone came to you wanting to know good software development practices, wouldn't you want to start with the basics of using code repositories, code review, style guides, and a discussion of waterfall v. agile? Before you started talking about how to manage a team of 500 developers? Yet on the cloud side, you act like it's unimportant whether Netflix does the former. As someone who would like to see much more enterprise cloud adoption, I see it as very important.
bmoyles
50%
50%
bmoyles,
User Rank: Apprentice
3/27/2013 | 6:12:17 PM
re: How Netflix Is Ruining Cloud Computing
So if someone wholesale adopted the entire Netflix stack without doing any sort of evaluation or due diligence to see whether the workflow it promotes makes sense within the context of their business and the applications it requires, Netflix should be responsible for that situation? Why is Netflix any more responsible than, say, the Apache Foundation for someone adopting Cassandra when MySQL was a better fit, or VMware selling vSphere to an organization whose applications don't lend themselves to virtualization, or PHP for existing at all? (Given that the 25,000 instances of cowsay was lost on you, I will point out: that last comment is sarcasm, and PHP, despite my personal feelings towards it, is used to solve many problems and is the right tool for many folks)

These tools are out there to promote what we think is *one* good way of doing things, and for our needs, they have worked phenomenally well. There are some organizations who may benefit from the whole stack, others may just need a piece, and other still might not find anything that fits.

Furthermore, this assertion that our process and tooling is only beneficial at large scale is a bit absurd. Does your infrastructure have the need to dynamically expand and contract capacity based on demand? Our tools *might* be a fit. Do you need the ability to build graceful degradation into your services? Our tools *might* be a fit. Do you need the ability to launch multiple instances of a given application and manage those instances as a unit? Our tools *might* be a fit.

Are they the One True WayGäó? Of course not, that would be silly to assert. Are they representative of a possible way that might work for your organization? Absolutely. Is there *something* in the suite of released tools that might benefit a variety of enterprises? I think so, but that call is ultimately up to the enterprise.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/27/2013 | 8:39:29 PM
re: How Netflix Is Ruining Cloud Computing
Well, in calling a contest "Fix the Cloud", it's a bit like saying "One True Way" (a better name might be, "Improve Our Proprietary Toolset"). From a future-looking standpoint, the Netflix cloud tools are less flexible than tools (like Scalr) that adopt abstraction layers for interacting with IaaS and that anticipate the future needs of working with multiple IaaS vendors (which, again, is just a matter of *when*, not *if*).

At the end of the day, what I'm most concerned about is promoting best-practices cloud implementations, because bad initial implementations will hurt cloud adoption significantly. Netflix's architecture and plans may be ideal for Netflix, but they're not an ideal reference architecture because so much focus in the Netflix architecture is on supporting Netflix's specific use case. For example, the focus on baking AMIs: it adds complexity (an additional step, additional storage costs, additional management requirements, additional testing, additional shipping of data cross region/cloud) because, for Netflix, launching 500 identical VMs at a time is common. But that's not common for the average cloud architecture. The additional complexity of baking AMIs--which is essentially required by the Netflix architecture--is unnecessary when your primary focus is high availability across multiple clouds/regions using a small number of each VM. This is similar to the use of something like an in-memory database: there are use cases where it's essential, but requiring an in-memory database in a general-purpose reference architecture doesn't make much sense.

I have never argued that Netflix wasn't providing a useful service--I am arguing that with great power comes great responsibility, and Netflix (and its defenders) seems uninterested in thinking about novices using its tools to adopt the cloud in ways that will be sub-par.
bmoyles
50%
50%
bmoyles,
User Rank: Apprentice
3/27/2013 | 9:30:36 PM
re: How Netflix Is Ruining Cloud Computing
You keep insisting that pre-baking AMIs is somehow adds complexity but have yet to articulate how. Please explain how pre-staging content and configuration on a fixed image prior to launch is any more complex than maintaining push/pull infrastructure and servers to dynamically turn running instances into services. I have no extra infrastructure to maintain to support configuration management other than version control. I have a process that reaps old, unused images. I have, at the end of a bake, an entire OS image that represents a point-in-time configuration of my application that I can refer to for triage. The absolute minimum bake script would be maybe 50 lines of bash? This is more complex....how?

Think of the rate at which the average enterprise is releasing their applications. How many AMIs is a smaller, slower-moving organization really going to produce in reality? Given the cost of S3 storage, is that really a significant cost? If it is...well... *baking is not for them.* Period. Full stop. End of story.

Do people managing 20-node clusters not have the need to roll out identical images too? Do smaller organizations have no need to dynamically manage capacity? What is it about baking that makes you think it's specifically intended for large launches? Every developer at Netflix uses the process, whether their autoscaling group is a single node or 1000 nodes.

Also please explain why you believe that the Netflix platform *depends* on baking in any way, shape, or form. It does not. We choose to bake and feel it is the best thing for us, but it's not required by any component in our infrastructure.

There is nothing whatsoever stopping folks from using Asgard to launch N instances that then have configurations applied to them post launch by Puppet, Chef, Saltstack, cfengine, etc. They just have to wait longer for the instances to become ready to take on traffic than they would if they baked ahead of time. That's the *major* difference.

The Netflix way of "adopting the cloud" is not sub-par in any way *for Netflix.* It's not for everyone, but it works and it works very well. I still fail to see how Netflix or any other organization providing open source software is at all responsible for any enterprise that blindly adopted tooling, process, policy, or anything without evaluating whether or not it made sense in the context of their business. The assertion that Netflix, by virtue of releasing its software and encouraging folks to help improve, extend, and *port it to other clouds*, is somehow beholden to organizations who have no ability to decide whether a solution is the right one is patently absurd.

Op ed and blogging does not disavow one from journalistic integrity. Each and every one of your points could have been addressed by engaging anyone at Netflix. Little things, like ensuring that someone tweeting about 25,000 cowsay AMIs isn't on the team that produced the tooling, or confirming that pre-staged AMIs are mandatory for playing in the ecosystem... Failing to follow through on that before posting a manifesto of FUD will be far more damaging to cloud computing than any contest could ever hope to be.

At this stage, though, we might as well be talking about guns or abortion or right- versus left-wing politics, as your insistence on regurgitating falsehoods underscores the fact that there is no shaking your preconceived notions and biases. I'm going to get back to the herd, 25,000 cows is a lot of cows.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/28/2013 | 2:06:38 AM
re: How Netflix Is Ruining Cloud Computing
Well, talk about emotive.

No one has addressed the points raised by my citing Adrian's quotes. I take from that that I am right, and Netflix simply refuses to consider any other vendor because of the same religious adherence to AWS that you're accusing me of having.

Ultimately, my piece raises two issues: (1) Single-vendor adherence, and (2) what's the best reference architecture for cloud deployments. As I said above, no one has disputed the adherence to AWS that I read from Adrian's quotes. And you and Adrian (and others) have said that Netflix's cloud architecture is probably not a good reference architecture for the cloud. So from a fundamental standpoint, we do not disagree. (You seem to want to argue amongst the trees; I'll concede the trees to you--my concern is with the forest).

One last point: you ask me to defend my claim that baking AMIs adds complexity--and immediately thereafter, you explain why it adds complexity (additional step, reaping, etc). Systems with fewer steps and less complexity are generally less fragile than those with more steps. The bake v. no bake decision (as I think Adrian addresses elsewhere on this page) is something that should be made on a per-use-case decision. The idea that baking is *always* superior to not baking is not true. (Again, this appears to be undisputed by Adrian).
jason833
50%
50%
jason833,
User Rank: Apprentice
3/30/2013 | 12:24:18 AM
re: How Netflix Is Ruining Cloud Computing
From where I stand, Joe's taking the name of the contest way too literally; hence, calling out Netflix for implying that the Cloud = AWS.

But are there other trees in the forest, or is it just a 1-tree forest with a bunch of shrubs. Google Compute Engine (GCE) is looking extremely promising, so I'm eagerly watching it grow into a tree.

As for bake v. no bake, it depends on many variables. How big is the infrastructure per service, how large is the change, how many iterations performed per... It's a moot point.
mtimc
50%
50%
mtimc,
User Rank: Apprentice
4/1/2013 | 1:06:53 PM
re: How Netflix Is Ruining Cloud Computing
Joe
Have you got any specific best practices in mind, beyond the obvious one of 'assume cheap, ephemeral hardware'? Which as significant changes on enterprise software. Followed by 'Continuous Delivery' (or deployment for some), so that you've got mechanisms for testing and replacing the components at low risk?

It seems to me that Netflix has done some sterling work in these dimensions, but that it's very unobvious to most enterprises that they are necessary underpinnings of moving to pay as you go infrastructure.

Can you be more specific in the abstractions that you have in mind?

I'd also add that Amazon does seem to be reacting to competition as price drops feel like they are accelerating, even if they are still nowhere near the cost advantage that AWS has according to James Hamilton.
jemison288
50%
50%
jemison288,
User Rank: Moderator
4/2/2013 | 11:00:16 AM
re: How Netflix Is Ruining Cloud Computing
I would put it this way (and I plan on writing in more detail on this): I think from an IaaS provider, you need the equivalent of EC2 (at least a few machine types), S3, EBS (snapshot/restore/detach/attach), and Security Groups from AWS. Then you should interact with that IaaS through an abstraction layer, not directly with their API, so you can switch to other, similar providers. (Netflix does argue that since they're at least--in some cases--using boto/similar libraries, it will be easier to add support for other clouds, but I am skeptical).

So, on that abstraction layer. Ultimately, from a software architecture standpoint, you can either start from a "I'm going to build a library that interfaces with X API", or you can start from, "I'm going to build a generalized set of interfaces to a certain type of service, and I'm going to translate from my generalized interfaces to specific implementations". I believe--and I think the evidence shows--if you start with the former, it's very hard to connect to other APIs that do similar things, just not in the exact same way. However, if you do the latter, and you are explicit from the beginning that you'll be translating from your outward-facing interfaces to other APIs, you'll have set the project up properly. These are really quite different software engineering projects, and I think that's why you hear the Netflix engineers really focus on the idea that other platforms will have to adopt the AWS API for things to work properly with their tools.

I view the multi-cloud issue as (a) critical, and (b) very hard, for the reasons you mentioned above. So much software uses RDBMS, and multi-master replication is generally speaking not the most reliable thing. Netflix has a good article about how they treat different types of data to handle their multi-region deployment, and I think we're going to have to move to architectures like that to take full advantage of the cloud.

In the end, I think the debate in these comments has more to do with us talking past each other than anything else (as Charles Babcock points out here somewhere). I think we all agree that Netflix has solved their particular use case in a great way, and that open sourcing their tools and giving speeches on their architecture is very useful. Netflix appears to view their AWS API lock-in as a necessary thing to keep for the future (and I don't see why they shouldn't work toward abstracting all of it) and Netflix appears to think that everyone approaching their tools will be capable of making an informed decision about the potential issues of using them (and I think that's a really bad assumption).
mtimc
50%
50%
mtimc,
User Rank: Apprentice
4/24/2013 | 2:03:01 PM
re: How Netflix Is Ruining Cloud Computing
Joe
Do you just have a minimal shim abstraction, then?

It seems to me that the AWS API *is* a good (enough) abstraction for now - it's so much richer than the competition and the experience of using it still too limited to establish a more abstract version. Provided your following a decent CD process, you'll have the tests in place to allow you to evolve the abstraction as you learn.

In my experience, abstraction shims can be useful, but they can also cost more than they're worth unless the problem domain is very well understood.

Discuss...
Joe Sondow
50%
50%
Joe Sondow,
User Rank: Apprentice
3/27/2013 | 5:43:23 AM
re: How Netflix Is Ruining Cloud Computing
Auto Scaling Groups.

I'll put aside the inflammatory, hyperbolic headline of the editorial for a moment, and talk about Auto Scaling Groups. Let's see how many times I can mention Auto Scaling Groups. Somebody count for me please.

At the core of Asgard's functionality is the Auto Scaling Group.

When Eucalyptus asked what they need to do in order to run Asgard against a Eucalyptus server, I told them they need to implement Auto Scaling Groups, and stub out a few other unimportant Amazon services Asgard currently expects to call. A few months later, they came back and said they were done. I asked if they implemented scaling policies. Yep. CloudWatch metrics? Yessir. Scheduled actions? You bet. Great! Let's finish making this thing flexible enough to use a Eucalyptus server. Someone still needs to add configurability to Asgard for regions, endpoints, instance types, application provider, and cloud API authentication. Cloud prize, anyone?

When OpenStack support consultants ask me how they can run Asgard against OpenStack, I tell them that first OpenStack needs to support the concepts that make Asgard useful, specifically Auto Scaling Groups. If you want to use Asgard without Amazon and without a cloud that has Auto Scaling Groups, then I really have to ask why. That's like using a food processor to open an envelope; you might get it to work, but to what end? There's maybe one screen in Asgard that might be useful for launching an instance without an Auto Scaling Group, but we don't use that screen much. Instead, I recommend choosing some implementation of Auto Scaling Groups, either through Scalr, Amazon, Eucalyptus or RightScale. The Auto Scaling Group serves to name and version a cluster, while associating it with an owner, and guaranteeing that the instances are homogeneous. The important part is the named group of instances of a single immutable image. The dynamic scaling part is gravy, although it does save you a lot of money.

As a partial substitute for the AWS Console, Asgard serves seven purposes for corporate Amazon customers, listed on the Netflix tech blog post where I first announced Asgard. (Google asgard tech blog). The purposes are: (1) Hide the Amazon keys, (2) Auto Scaling Groups, (3) Enforce conventions, (4) Logging, (5) Integrate systems, (6) Automate workflow, (7) Simplify REST API. When and if Amazon adequately addresses all seven of those issues in their own console, then I will gleefully recommend that Netflix deprecate Asgard and start using the AWS console instead. Then I'll go write some movie-related software instead. However, I'm not holding my breath. Amazon has a lot of other things to consider beyond supporting the cloud model Netflix has chosen. My prediction is that Asgard will remain a reasonable option for customers of cloud providers that have Auto Scaling Groups, starting with Amazon.

Is the publicity of Asgard putting pressure on cloud providers to implement both Auto Scaling Groups and usable graphic interfaces for configuring those Auto Scaling Groups? I hope so. That's one of the reasons I wanted to open source Asgard. If nobody can figure out how to use Auto Scaling Groups, then no one will use them. Then Amazon is less likely to add them to their console and less likely to augment them to be more useful, and Google is less likely to implement them. Auto Scaling Groups are great. Let's use them. Let's tell more cloud providers to provide them.

Will another company do as Eucalyptus did, and clone enough parts of the Amazon API to get free benefit from our tools? That would be good. Remember, Eucalyptus did most of that work before Amazon even talked to them. If cross-cloud-provider portability is your focus, my advice would be to add to Eucalyptus' open source implementation and make it plug into a dozen other cloud vendors the way it plugs into any data center. Personally I'm more interested in using so many isolated AWS regions that I don't need to worry about any one AWS system having a problem.

Now, let's talk a little more about AMIs.

Relying on a Chef/Puppet configurator for every production instance launch is not a good idea. It's a really bad idea. I don't why anyone would regard deploy-time configuration as something new and good, while regarding pre-baked image launching as something old and bad. It's the other way around. You might be used to the idea of deploy-time configuration, but it's still a bad idea. It's an unnecessary risk. The point of Aminator is to give people a robust way to stop thinking in that old school way. I want people to start using Chef at build time, not deploy time. Use Chef with Aminator to create a complete image of the latest version of your application. Then know with certainty that every instance of that AMI will be identical in the development, test, staging, and production environments, in multiple redundant regions across four continents, even if the network fails during instance startup, even if the Chef server is getting upgraded or is falling over one day, even if a second deployment of the image happens months later. All the instances will be homogeneous within an Auto Scaling Group, all the time, even at large scale.

For the past 9 months, Aminator was the missing piece in the story of Asgard's ease of use. Now that there is a convenient way to produce a new AMI for each software build, it should be easier for people to use Asgard and Auto Scaling Groups for deployments without needing to rely on a highly available production deploy-time Chef server. If these resiliency concepts can be offered by more cloud providers, so much the better. I don't think that's ruining the cloud. I think that's promoting good patterns for tomorrow's cloud.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/27/2013 | 10:23:33 AM
re: How Netflix Is Ruining Cloud Computing
Thanks for the response. Unfortunately, the only company with authorized rights to the AWS API (other than AWS) is Eucalyptus, so what you make sound so easy in your response is so fraught from a legal perspective that you're not going to find any other provider doing it. Perhaps if AWS were less proprietary and more willing to contribute to the overall community, they would allow providers to implement it as their own APIs. Perhaps you could help put that pressure on them? Because once the AWS API can be used as an open standard, then Netflix's tools will instantly have a much bigger audience.

On AMIs: Your assertion that using Chef/Puppet for every launch "is not a good idea" assumes that you've got a lot of VM launches that will be identical (machine, cloud). Again, don't look at this from "what does Netflix do internally"; look at it from "general enterprise cloud adoption". By using an AMI-centric model of the world, you're (a) adding overhead to each release, (b) creating a management/cleanup/storage situation that you would not have otherwise, and (c) requiring yourself to treat launches in different regions/clouds differently--including verifying that you have the right images in the right places properly baked and ready to go. In contrast, using Chef/Puppet on every launch avoids every single one of those problems, and thus gives you much more flexibility. The cost of Chef/Puppet on each launch is that it adds (d) overhead (time, bandwidth) and (e) some additional level of fragility (how much depending upon where you're pulling files).

Your assertion that using Chef/Puppet on each launch is "not a good idea" shows how Netflix-centric your world is. For many, many people, Chef/Puppet on every launch is a much better business and technological decision than rolling AMIs for each release because the pain of (a), (b), and (c) is greater than the pain of (d) and (e). In fact, the fragility of (a) + (b) + (c) can be significantly greater than the fragility of (e).

Ultimately, this is not a referendum on "how Netflix should run its cloud architecture". This is a referendum on whether Netflix should have a responsibility to the cloud computing world to help novices understand best practices in running clouds versus running a contest that is more likely to promote sub-par use of the cloud.
gregdek
50%
50%
gregdek,
User Rank: Apprentice
3/27/2013 | 3:33:23 PM
re: How Netflix Is Ruining Cloud Computing
"Unfortunately, the only company with authorized rights to the AWS API (other than AWS) is Eucalyptus, so what you make sound so easy in your response is so fraught from a legal perspective that you're not going to find any other provider doing it."

Except, of course, that other providers *are* doing it right now, and have been doing it for years. Cloudstack has Cloudbridge. Red Hat has Deltacloud/Aeolus. And it's all open source. Sure, we're moving faster down this path at Eucalyptus -- but it's the exact same path.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/27/2013 | 8:55:43 PM
re: How Netflix Is Ruining Cloud Computing
There is no IaaS provider live with an AWS API layer that supports it with an SLA because of the legal issues. I'm not just making this up; it's a significant issue that most cloud consultants who work for large organizations are worried about (e.g., DoD).

When I was in college, every college student I know used Napster to pull down music. That didn't make that activity legal either.
adrianco
50%
50%
adrianco,
User Rank: Apprentice
3/27/2013 | 5:02:38 PM
re: How Netflix Is Ruining Cloud Computing
Both ways of creating images are valid and tooling should be used to automate every step of the build process. There should be no hand crafted images. Every image should be traceable to the bits it was built from. That's the best practice of cloud.

The chef at runtime approach works fine at small scale but breaks horribly at large scale. When you have lots of developers changing things at once you want to build using the latest bits and freeze that build for test and deployment. For availability you would need multiple distributed Chef servers, but you then have to guarantee that they are always in sync, which is one of the hard problems of distributed computing. Avoiding that problem has value.

Baking AMIs is wasting a cheap resource, and we have tooling to clean up the leftovers. These are implementation choices that should be made appropriately to the situation. The NetflixOSS PaaS is interesting to many enterprises who do have large scale problems, and who find that other PaaS solutions are currently optimized for startups and small scale applications.

If you only have 10s of instances, NetflixOSS is likely to be overkill. If you have 100s it becomes useful, with 1000s it's essential and 10,000s it's probably the only game in town at the moment. With 100,000s you are Facebook or Google anyway...
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/27/2013 | 8:57:02 PM
re: How Netflix Is Ruining Cloud Computing
Agreed. This would be an excellent forward to have to people looking at using the NetflixOSS tools.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/26/2013 | 10:55:03 PM
re: How Netflix Is Ruining Cloud Computing
Adrian--

Here's the tree response.

Cloud 1.0 vs. 2.0:
You say, "The specific IaaS provider used underneath, and whether you do this with public or private clouds is irrelevant to the architectural constructs we've explained." But of course, the Netflix contest has to do with your tools, not "architectural constructs" per se. And your tools are absolutely tied to a specific IaaS provider. And from an architectural perspective, while Netflix has done some awesome things, I think Zynga's architecture (including what they pay for what they get) is much more likely to be what best-practice enterprise cloud architectures look like in 5+ years. I don't know many cloud architects who are aware of the difference between Zynga and Netflix who would pick Netflix's implementation over Zynga's--again, largely because of the multi-cloud capabilities.

Outages:
My point in bringing up the outages was not to imply that they were international or fatal; it was to point out that Netflix's cloud architecture is not perfect (not that any cloud architecture is, but always good to point this out), and that one can tie at least one outage to Netflix's specific architectural decision to embrace a proprietary service (ELB) when other, non-proprietary, more resilient options were available. I'm happy that Netflix won't repeat an outage due to that specific proprietary service, but the overall philosophy of choosing AWS services over open options that are more flexible and more resilient remains.

Portability:
As long as you continue to force Netflix to use new and expanded Amazon-provided services over other options, you're creating a moving target that no other vendor will hit. The better path to take is: how should organizations design their architectures so that they can maintain portability and interoperability across multiple vendors, whereever possible? If, today, we wait for Google Compute Engine to be more widely available and tested, then shouldn't we be moving to abstraction layers for API communication, instead of doubling down and adopting more and more proprietary APIs? Perhaps AWS will release their API to the world and allow all businesses to use it openly, but they haven't yet, and so it's a very risky move to bet an architecture on AWS and any vendors (e.g., Eucalyptus) that AWS will bless. Perhaps what you're saying is that you'd be happy to see abstraction layers in the Netflix tools for working with other clouds, but *you're not actually saying that*. Please say that.

Edda:
Thanks for the details on Edda; my knowledge was from reading what was easily available on github: "Why did we create Edda? ... if we see a host with an EC2 hostname that is causing problems on one of our API servers then we need to find out what that host is and what team is responsible, Edda allows us to do this." Edda sounds like a great tool to take multi-cloud--would you considering suggesting that as a theoretically good project for the contest (no guarantees on prizes)?

AMInator:
Your explanation of using Chef with AMInator makes a lot of sense in the "500 simultaneous instances" use case. Which is--you would admit--not a common circumstance amongst the people who use/will be using your cloud tools. And unfortunately, your first happy user of AMInator (on Twitter, at least) made over 25,000 Ubuntu AMIs with it--can you tell me why that would ever be a good architectural decision? AMInator strikes me as a tool like PHP or a GOTO statement--there are places where you should probably use them, but it's hard to argue that they should be part of any kind of "best practices" decision.

Cloud Prize:
The fact that only one out of ten prizes involves portability, and the fact that you take such an expansive view of portability to include adding language support to an existing tool (which has NOTHING to do with cloud portability!), shows that you really think that cloud portability unimportant to Netflix. If Netflix wants to make that business decision, then fine. But I would argue that Netflix is a role model in the world, and has a lot of ears, and that it's just irresponsible for Netflix to lead the rest of the world on the same path.

To the extent that Netflix is trying to exploit open source in the same way many companies do--to share code in exchange for getting additional development for free--I have no issues. Go for it. But I have a problem in the way that Netflix's tools and architectural decisions are taken as THE reference architecture. I write here, and I wrote the piece, not to try to convince you, Adrian, to change the way Netflix does things. I would like you to the run the contest in such a way that it promotes portability and interoperability and make the judging panel less AWS-centric. But beyond that, I'm really writing these for those people out there considering whether Netflix's cloud architecture is something they should copy verbatim. (Don't!)

Heidi Roizen, an entrepreneur-turned-VC, put it this way: "I don't ask 'what happens if?' ... I ask 'what happens when?'"--meaning that there are certain things that we know will happen, and if we aren't thinking about them and planning for them, we're not thinking strategically enough. (One of her examples: "what happens when we have self-driving cars?") It is a certainty that we will have viable IaaS competitors to AWS. But the attitude that is embedded in the Netflix cloud tools--and from what I see of the contest today--is one that essentially says, "we will look nowhere but AWS." And there is no thing that has been said in response to my piece that says otherwise. In fact, you don't even address my quoting of you in different places where you have excluded other options by fiat, regardless of price or functionality.

And that is the problem.
adrianco
50%
50%
adrianco,
User Rank: Apprentice
3/27/2013 | 12:11:57 AM
re: How Netflix Is Ruining Cloud Computing
You're using some emotive language here: "as long as you continue to force Netflix to use new and expanded Amazon-provided services over other options".

What actually happens is that Netflix has various problems to solve, and we do the usual make/buy evaluations, and sometimes we make it ourselves (e.g. Asgard over the AWS Console or other options) and sometimes we get vendors to build them. Part of that process is to work with AWS to build things we would like to use, but are of general use, so we don't want to build them ourselves. Part of the reason for releasing NetflixOSS is to make explicit to other cloud vendors the feature set and options that we have found useful, to see if they are also interested in responding.

You said "Perhaps AWS will release their API to the world and allow all businesses to use it openly, but they haven't yet, and so it's a very risky move to bet an architecture on AWS and any vendors (e.g., Eucalyptus) that AWS will bless."

For a start there are very many vendors who have implemented parts of the AWS API, and Eucalyptus have licensed the API so that they have access to the test suites. Nothing stops other vendors from doing the same.

It would only be a risky move to bet on an architecture that might go away. Betting on the industry leader with the dominant ecosystem looks like the lowest risk option to me.

Still, everyone is free to ignore NetflixOSS, there are plenty of other cloud architectures. But Netflix is innovating and scaling faster than the competition because we are also leveraging the innovation and scale of AWS.
gregdek
50%
50%
gregdek,
User Rank: Apprentice
3/27/2013 | 12:25:32 AM
re: How Netflix Is Ruining Cloud Computing
"The fact that only one out of ten prizes involves portability, and the fact that you take such an expansive view of portability to include adding language support to an existing tool (which has NOTHING to do with cloud portability!), shows that you really think that cloud portability unimportant to Netflix."

The fact that Adrian encourages us at Eucalyptus and our friends at Cloudstack to tackle the portability problem head-on shows otherwise.
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/27/2013 | 8:58:40 PM
re: How Netflix Is Ruining Cloud Computing
Right--because you have the right to use the AWS API.

Do you really not see a difference between the NetflixOSS approach to interacting with the cloud compared to the Scalr approach?
bmoyles
50%
50%
bmoyles,
User Rank: Apprentice
3/27/2013 | 3:27:20 AM
re: How Netflix Is Ruining Cloud Computing
This is bananas...

"And unfortunately, your first happy user of AMInator (on Twitter, at least) made over 25,000 Ubuntu AMIs with it--can you tell me why that would ever be a good architectural decision? AMInator strikes me as a tool like PHP or a GOTO statement--there are places where you should probably use them, but it's hard to argue that they should be part of any kind of "best practices" decision."

No, that was me. Not an aminator user, but one of the aminator *authors*. Feel free to verify that both my Twitter and Github accounts align and feel free to observe aminator's commit history. Heck, look at my Twitter profile, where it is very clear that I...work for Netflix. If you took *anyone's* offhand comment about creating *twenty five THOUSAND* AMIs (let alone one of the people working on the tooling to do so), all with the application 'cowsay' as their primary component as being anything other than a joke, I don't know what any of us can do for you to help you understand motivations or intentions (unless you are aware of any large-scale talking ASCII cow clusters, in which case I stand corrected).

"One of the reasons why Netflix is now choosing Python is because the generalized Python developer writes consistent and good code. (We chose Python for the same reasons you did). But to someone who has no idea what a good cloud deployment looks like, the way AMInator sits out there--you're going to see a lot more people like the guy super-psyched to have built 25,000 AMIs over Twitter."

We do not choose technologies based on what prospective developers *might* do with it, like write "consistent and good" code (and the assertion that Python, by some magical virtue, makes good programmers is hogwash. Good developers write consistent and good code regardless of the language, and many people write bad Python with ease.) We choose technologies that fit the job and the situation. Given that a) aminator was intended to be run ad-hoc or from some other automation, b) there is a fair amount of Python experience amongst Netflix employees, and c) languages such as Python (and Ruby and Perl and ...) lend themselves to rapid iterative development, we felt it was the right tool for the job. While I personally enjoy using Python, there is nothing about aminator that couldn't have been done with Ruby, Perl, TCL, PHP, Java, Groovy, Scala, or heck, bash (which is what aminator's ancestor was developed with).

Aminator was built to be modular, and any of its 5 major components (at this point) can be replaced to work with any system you can conceive of. There's no reason a set of plugins couldn't be developed that produced images for Windows on Azure, or local disk images for use with VirtualBox. What we provided was a framework and *our implementation* which *naturally* services our needs. Folks are free to use it as-is, or they can take what we have and replace parts with what works for them. I really hope they do, too. I'd love to see it produce Windows images, FreeBSD images, and so on.

More documentation on how to use and extend aminator is on its way, and Netflix staff is in #netflixoss on irc.freenode.net fielding questions as they come in. You too are welcome to join and ask questions before posting articles, in case that wasn't clear :)
jemison288
50%
50%
jemison288,
User Rank: Moderator
3/28/2013 | 2:23:15 AM
re: How Netflix Is Ruining Cloud Computing
Your attitude toward Python here is quite different from how you appear to approach it in other spaces (the Netflix techblog article, Pycon, etc); and it's very interesting to me in light of the rigid adherence to AWS over all other IaaS options, seemingly forever. Where is the same level of skepticism and evaluation on AWS that you appear to have toward programming languages? Again, no one is addressing Adrian's comments that he is uninterested in any other IaaS vendor period/no exceptions. If that's not the case, then it would be wonderful to hear. Otherwise, it just stands out as a very different attitude and decision than what you're doing elsewhere in your technology choices.
mtimc
50%
50%
mtimc,
User Rank: Apprentice
4/25/2013 | 9:19:57 AM
re: How Netflix Is Ruining Cloud Computing
Joe

Do you mean that Adrian's changed his position from where it was on slide 20 of:http://www.slideshare.net/adri... (March last year)?

Compared to other approaches to portability that I've encountered, NF seems to be quite grown up and pragmatic and actively worried about lock-in.

Can you point me at a relevant citation?
cbabcock
50%
50%
cbabcock,
User Rank: Strategist
3/27/2013 | 1:26:58 AM
re: How Netflix Is Ruining Cloud Computing
I agree with Joe Emison when he upholds cross cloud mobility and multi-cloud tools as the ultimate goal. I agree with Adrian Cockcroft when he pursues innovation and fresh ideas for the AWS context... in which Netflix currently and for the foreseeable future operates. Each has a different purpose behind his argument, and so their points are to some extent sailing past each other without registering and certainly without scoring, I don't like to see too much judgmental-ness applied to other people's architecture. The judgement should be applied to our own efforts and let the other fellow pursue his initiative to the max, even if it's initially seemed to fail to meet one or more of our ultimate, far sighted standards. The cloud is young and it's impossible to say how some seemingly small or narrow-minded effort might grow legs and lead to long term gains for everybody. With that said, both sides have presented a case well and I've learned from this debate. Charlie Babcock, senior writer, InformationWeek
middleageman
50%
50%
middleageman,
User Rank: Apprentice
3/28/2013 | 6:17:33 PM
re: How Netflix Is Ruining Cloud Computing
"I'm pointing out that this roots you in 2008, and ignores everything that AWS has developed in response to the needs of their customers in the following five years." I read this wonderful dialog and thank you for it. I agree that portability and cross platform tolerance it tantamount to success for us, the users. for you, the producers, kudos for even considering our needs above our $'s. (which follow anyway) now, about self driving cars and regional outages, which regions? i need to know.
jason833
50%
50%
jason833,
User Rank: Apprentice
3/30/2013 | 12:52:36 AM
re: How Netflix Is Ruining Cloud Computing
AWS got an early head start in IaaS offering, and built a massive eco-system with all the building blocks that no others can follow even to this day. I just don't see why anyone would want invest the time and money on cross cloud mobility and multi-cloud tools at this stage.
jemison288
50%
50%
jemison288,
User Rank: Moderator
4/2/2013 | 11:01:08 AM
re: How Netflix Is Ruining Cloud Computing
I assume you do not work in a risk management or cost management role, then?
jason833
50%
50%
jason833,
User Rank: Apprentice
4/2/2013 | 11:00:00 PM
re: How Netflix Is Ruining Cloud Computing
That's an interesting assumption. As matter of fact, I do which is why I chose AWS as my IaaS provider.

AWS has 3 independent Regions in the US (risk management), and it's cheaper to deploy multi-regions using the same toolsets that works for all, then spending time and efforts building your toolsets to work with other IaaS providers (cost management). In addition, combining both, you get stability and ease of management by reducing complexity of your Infrastructure as a whole.

With that said, I'm eagerly waiting for Google Compute Engine to grow into a "tree", as I've read favorable review on its limited preview. Google is probably the only powerhouse out there who can compete with Amazon head-on in terms of scalability and technologies. Thoughts?
David Berlind
50%
50%
David Berlind,
User Rank: Apprentice
4/2/2013 | 10:49:41 PM
re: How Netflix Is Ruining Cloud Computing
Just a heads up to everyone on this thread that Rackspace CTO John Engates has chimed into the Netflix debate with an op-ed piece on InformationWeek.com. You can find it here: http://www.informationweek.com...
Decision Consultants
50%
50%
Decision Consultants,
User Rank: Apprentice
4/30/2013 | 3:54:40 PM
re: How Netflix Is Ruining Cloud Computing
Great interesting article. Interested to know who will win the contest.
Google in the Enterprise Survey
Google in the Enterprise Survey
There's no doubt Google has made headway into businesses: Just 28 percent discourage or ban use of its productivity ­products, and 69 percent cite Google Apps' good or excellent ­mobility. But progress could still stall: 59 percent of nonusers ­distrust the security of Google's cloud. Its data privacy is an open question, and 37 percent worry about integration.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - August 20, 2014
CIOs need people who know the ins and outs of cloud software stacks and security, and, most of all, can break through cultural resistance.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.