Netflix gained experience in the cloud by using it in 2009 to encode the digital versions of movies that it was adding to its library. It used thousands of instances, or virtual machines, in EC2 to execute the task as a batch job "and get more movies online," he said. Netflix is also a user of petabytes of Amazon's S3 storage.
It also began moving its Web site into EC2, a process that is largely completed, he said.
At no time did Cockcroft express dissatisfaction with Amazon as a cloud service provider, although he did acknowledge several challenges. Netflix chose Amazon over other cloud providers based on its size, ability to grow its services quickly, and its cloud feature set.
Among the challenges was Amazon's EC2's Elastic Load Balancer, which he termed as possessing "too many limits" for the scale at which Netflix wished to operate.
In addition, SimpleDB is good for dozens or hundreds of gigabytes of data, but Netflix needs it to handle terabytes at a time. That need "was beyond its sweet spot," although Netflix found SimpleDB living up to its name as an easy-to-use system.
Netflix built "a large tier" of data cache management in front of SimpleDB to handle more data. It used the open source system memcached to do so.
Cockcroft also said the performance of EC2's Elastic Block Store storage system for running applications was "slow and too inconsistent" for Netflix's purposes and it substitutes its own system built with the NoSQL system Cassandra.
The move to the cloud prompted the abandonment of many former standard practices in the data center. Netflix, for example, no longer uses a change management database because they work best with relatively static systems that change infrequently. The fluid, rapidly changing alignment of virtual machines in the cloud means Netflix had to impose a new method of managing changes.