Cloud // Infrastructure as a Service
News
4/10/2014
11:35 AM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Netflix Cloud Architect: How To Succeed As Disruptor

Adrian Cockcroft, Netflix's former chief cloud architect, outlines how a changed business flowed out of the cloud.

8 Data Centers For Cloud's Toughest Jobs
8 Data Centers For Cloud's Toughest Jobs
(Click image for larger view and slideshow.)

Adrian Cockcroft, former chief cloud architect for Netflix, recently described an "evolution of business" away from familiar processes into a more digital world, where integration of teams and speed of development decide life and death for companies. During seven years at Netflix, Cockcroft became one of the chief advocates and explainers of what it means to move a business into the cloud. He was a frequent speaker at shows like Cloud Connect, and wrote about key cloud management techniques in his aptly named Adrian Cockcroft's Blog.

During his tenure, Netflix halted construction of its own datacenters in 2008 and moved its customer-facing operations onto Amazon Web Services. The move coincided with Netflix's decision to become a video-streaming business to a variety of consumer devices as well as a shipper of DVDs. In the process, it rapidly grew its customer base by millions and launched new ways of finding and ordering films online.

At the end of 2013, Cockcroft left Netflix to become a technology fellow at venture capital firm Battery Ventures. He spoke for 40 minutes April 1 at the InformationWeek Conference, collocated this year with UBM Tech's Interop Las Vegas. Cockcroft is a former distinguished engineer at Sun Microsystems in high-performance computing and a co-founder of eBay Research Labs.

[Want to learn more about Adrian Cockcroft's influence on cloud use? See How Netflix Is Ruining Cloud Computing.]

The changes wrought by digital services replacing physical ones have led to changes in how businesses must organize their staffs, conduct IT, and produce software, he said at the InformationWeek Conference. Many of the techniques he spearheaded at Netflix supported streaming services as a replacement for shipping DVDs. He said the only datacenter capacity that Netflix retained after it moved into the cloud supports the systems for DVD shipping and billing. Everything else has moved onto AWS virtual servers.

He didn't use the term "agile" to describe Netflix's software development, but the speed and close collaboration of developers and business managers he described would easily fit into many agile-inspired processes. Only at the end of his talk did he use "agile" to describe the state of the corporation as a whole, if it follows his recommendations.

Adrian Cockcroft(Image: Battery Ventures)
Adrian Cockcroft
(Image: Battery Ventures)

The old process -- laboriously defining a software project, ordering hardware to support it, coding it over a period of months or a year, and finally launching an application into production -- takes too long for today's pace of business, he warned. Aiming merely to "optimize a system," making it cheaper, faster, and better, is no longer sufficient.

"There's a real sense of innovation," in a Netflix-style approach. "There's an evolution of products from custom to something more like utilities," that can be periodically added to and built out, he said. This type of development "is usually trying to invent something people have never done before."

Cockcroft then launched into the heart of his talk. "Let me tell you briefly what I learned at Netflix. I was there seven years, right from the beginning when streaming started. The first thing we learned, the goal of Netflix [in keeping up with] the growth that was driving all the things that Netflix could do, was how fast can we move? Netflix at the time was a very small player in the market. The competitive advantage that we had was to move faster and learn faster and get things done more quickly."

But doing things faster meant speeding up decision-making and producing software to support new services. To do that, Netflix had to "remove the friction from product development." Friction is produced when designers of a product talk to product management and engineering teams to describe what must be done, then send each team off to do its own work. Incomplete communication, varying interpretations of requirements, and changing requirements in a changing marketplace all lead to friction.

A company must ask, "What are the handoffs, what are the things that slow you down? We put a high-trust, low-process environment in place with few handoffs between teams. We consolidated teams... We put things together. We removed process debt (overhead) completely," he recalled.

Designers, developers, and product managers were "consolidated" into single teams, with a small coding team responsible for the software from conception through deployment as a production system. It's a key DevOps goal to get developers to spend more time listening to the operations side of the house. At Netflix, developers became the operations side of the house. They continued to have responsibility for the software after it went into production.

Netflix created "a very developer-oriented culture," Cockcroft continued. "We give freedom to employees to do things, but they're responsible for them. [Netflix executives] are just incredibly insistent, you have the freedom to do things. If employees do something stupid, they generally learn from it. If they consistently do stupid things, then you have to get rid of them. There isn't a sense you can do anything you want, that you have tenure... You're free, but you're responsible."

Pushing responsibility down several levels allowed Netflix to produce new services quickly and try them out with customers rather than spending months building something that customers may not want. Cockcroft described it has a virtuous cycle of develop, release, test, go back to development again.

"The question is, 'How fast can you go around that cycle?' because that's how you learn something. We were learning something new all the time," he said, as Netflix discovered that customers wanted to download movies to game consoles and tablets, as well as to their computers.

Going around the customer test cycle requires "high trust" among team members and "low process," or few slowdowns, because tasks need to

Next Page

Charles Babcock is an editor-at-large for InformationWeek, having joined the publication in 2003. He is the former editor-in-chief of Digital News, former software editor of Computerworld and former technology editor of Interactive Week. He is a graduate of Syracuse ... View Full Bio

Previous
1 of 2
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
ChrisMurphy
50%
50%
ChrisMurphy,
User Rank: Author
4/11/2014 | 9:02:11 AM
Re: Was the cloud the cause or just an enabler?
Adrian's presentation was very thought provoking, especially coming right after General Motors CIO Randy Mott's session that made the case that cloud services might get you EVEN with your competitors, and ahead of where you are today, but it will never give you competitive advantage over competitors. The two sessions had a lot of people talking, since they offered such different ends of the spectrum. 
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
4/10/2014 | 2:37:34 PM
Was the cloud the cause or just an enabler?
Netflix' move to the cloud coincided with its disruption of the home movie rental business. The cloud, if not the cause of the disruption, was an enbler. Netflix had to convert analogue content on EC2, store it and then stream the new digitized content to millions of customers. The power of the cloud was available for each step that moved Netflix away from shipping DVDs.

 
Multicloud Infrastructure & Application Management
Multicloud Infrastructure & Application Management
Enterprise cloud adoption has evolved to the point where hybrid public/private cloud designs and use of multiple providers is common. Who among us has mastered provisioning resources in different clouds; allocating the right resources to each application; assigning applications to the "best" cloud provider based on performance or reliability requirements.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - August 20, 2014
CIOs need people who know the ins and outs of cloud software stacks and security, and, most of all, can break through cultural resistance.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.