Netflix Cloud Architect: How To Succeed As Disruptor - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Cloud // Infrastructure as a Service
11:35 AM
Connect Directly

Netflix Cloud Architect: How To Succeed As Disruptor

Adrian Cockcroft, Netflix's former chief cloud architect, outlines how a changed business flowed out of the cloud.

8 Data Centers For Cloud's Toughest Jobs
8 Data Centers For Cloud's Toughest Jobs
(Click image for larger view and slideshow.)

Adrian Cockcroft, former chief cloud architect for Netflix, recently described an "evolution of business" away from familiar processes into a more digital world, where integration of teams and speed of development decide life and death for companies. During seven years at Netflix, Cockcroft became one of the chief advocates and explainers of what it means to move a business into the cloud. He was a frequent speaker at shows like Cloud Connect, and wrote about key cloud management techniques in his aptly named Adrian Cockcroft's Blog.

During his tenure, Netflix halted construction of its own datacenters in 2008 and moved its customer-facing operations onto Amazon Web Services. The move coincided with Netflix's decision to become a video-streaming business to a variety of consumer devices as well as a shipper of DVDs. In the process, it rapidly grew its customer base by millions and launched new ways of finding and ordering films online.

At the end of 2013, Cockcroft left Netflix to become a technology fellow at venture capital firm Battery Ventures. He spoke for 40 minutes April 1 at the InformationWeek Conference, collocated this year with UBM Tech's Interop Las Vegas. Cockcroft is a former distinguished engineer at Sun Microsystems in high-performance computing and a co-founder of eBay Research Labs.

[Want to learn more about Adrian Cockcroft's influence on cloud use? See How Netflix Is Ruining Cloud Computing.]

The changes wrought by digital services replacing physical ones have led to changes in how businesses must organize their staffs, conduct IT, and produce software, he said at the InformationWeek Conference. Many of the techniques he spearheaded at Netflix supported streaming services as a replacement for shipping DVDs. He said the only datacenter capacity that Netflix retained after it moved into the cloud supports the systems for DVD shipping and billing. Everything else has moved onto AWS virtual servers.

He didn't use the term "agile" to describe Netflix's software development, but the speed and close collaboration of developers and business managers he described would easily fit into many agile-inspired processes. Only at the end of his talk did he use "agile" to describe the state of the corporation as a whole, if it follows his recommendations.

Adrian Cockcroft(Image: Battery Ventures)
Adrian Cockcroft
(Image: Battery Ventures)

The old process -- laboriously defining a software project, ordering hardware to support it, coding it over a period of months or a year, and finally launching an application into production -- takes too long for today's pace of business, he warned. Aiming merely to "optimize a system," making it cheaper, faster, and better, is no longer sufficient.

"There's a real sense of innovation," in a Netflix-style approach. "There's an evolution of products from custom to something more like utilities," that can be periodically added to and built out, he said. This type of development "is usually trying to invent something people have never done before."

Cockcroft then launched into the heart of his talk. "Let me tell you briefly what I learned at Netflix. I was there seven years, right from the beginning when streaming started. The first thing we learned, the goal of Netflix [in keeping up with] the growth that was driving all the things that Netflix could do, was how fast can we move? Netflix at the time was a very small player in the market. The competitive advantage that we had was to move faster and learn faster and get things done more quickly."

But doing things faster meant speeding up decision-making and producing software to support new services. To do that, Netflix had to "remove the friction from product development." Friction is produced when designers of a product talk to product management and engineering teams to describe what must be done, then send each team off to do its own work. Incomplete communication, varying interpretations of requirements, and changing requirements in a changing marketplace all lead to friction.

A company must ask, "What are the handoffs, what are the things that slow you down? We put a high-trust, low-process environment in place with few handoffs between teams. We consolidated teams... We put things together. We removed process debt (overhead) completely," he recalled.

Designers, developers, and product managers were "consolidated" into single teams, with a small coding team responsible for the software from conception through deployment as a production system. It's a key DevOps goal to get developers to spend more time listening to the operations side of the house. At Netflix, developers became the operations side of the house. They continued to have responsibility for the software after it went into production.

Netflix created "a very developer-oriented culture," Cockcroft continued. "We give freedom to employees to do things, but they're responsible for them. [Netflix executives] are just incredibly insistent, you have the freedom to do things. If employees do something stupid, they generally learn from it. If they consistently do stupid things, then you have to get rid of them. There isn't a sense you can do anything you want, that you have tenure... You're free, but you're responsible."

Pushing responsibility down several levels allowed Netflix to produce new services quickly and try them out with customers rather than spending months building something that customers may not want. Cockcroft described it has a virtuous cycle of develop, release, test, go back to development again.

"The question is, 'How fast can you go around that cycle?' because that's how you learn something. We were learning something new all the time," he said, as Netflix discovered that customers wanted to download movies to game consoles and tablets, as well as to their computers.

Going around the customer test cycle requires "high trust" among team members and "low process," or few slowdowns, because tasks need to

Next Page

Charles Babcock is an editor-at-large for InformationWeek and author of Management Strategies for the Cloud Revolution, a McGraw-Hill book. He is the former editor-in-chief of Digital News, former software editor of Computerworld and former technology editor of Interactive ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Author
4/11/2014 | 9:02:11 AM
Re: Was the cloud the cause or just an enabler?
Adrian's presentation was very thought provoking, especially coming right after General Motors CIO Randy Mott's session that made the case that cloud services might get you EVEN with your competitors, and ahead of where you are today, but it will never give you competitive advantage over competitors. The two sessions had a lot of people talking, since they offered such different ends of the spectrum. 
Charlie Babcock
Charlie Babcock,
User Rank: Author
4/10/2014 | 2:37:34 PM
Was the cloud the cause or just an enabler?
Netflix' move to the cloud coincided with its disruption of the home movie rental business. The cloud, if not the cause of the disruption, was an enbler. Netflix had to convert analogue content on EC2, store it and then stream the new digitized content to millions of customers. The power of the cloud was available for each step that moved Netflix away from shipping DVDs.

InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

10 Things Your Artificial Intelligence Initiative Needs to Succeed
Lisa Morgan, Freelance Writer,  4/20/2021
Tech Spending Climbs as Digital Business Initiatives Grow
Jessica Davis, Senior Editor, Enterprise Apps,  4/22/2021
Optimizing the CIO and CFO Relationship
Mary E. Shacklett, Technology commentator and President of Transworld Data,  4/13/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Flash Poll