Netflix goes all-AWS as it abandons in-house data centres

Netflix completes shift from in-house IT as it goes 100 per cent to the Amazon cloud

Films and television streaming website Netflix has abandoned in-house data centres and shifted its entire service over to the Amazon Web Services (AWS) cloud.

The move marks the end of a seven-year effort by Netflix to migrate its IT to the public cloud platform, and perhaps provides a pointer to the future of corporate IT.

The majority of the company's systems were shifted to AWS before 2015, but Netflix needed more time to securely move its billing infrastructure and customer and employee data management systems onto the cloud platform.

Netflix said that it opted for AWS as it has the broadest set of features, services and scale that can serve its international customer base.

"We rely on the cloud for all of our scalable computing and storage needs, our business logic, distributed databases and big data processing/analytics, recommendations, transcoding and hundreds of other functions that make up the Netflix application," Netflix said.

The migration means that Netflix no longer needs to run its own data centres or manage the servers in them, resulting in a cost benefit and the far greater scale offered by the global reach of AWS.

The move to the cloud was prompted by an increase in Netflix's user base, which would have seen the company struggle to deliver services had it not reworked its infrastructure.

"Supporting such rapid growth would have been extremely difficult from our own data centres; we simply could not have racked the servers fast enough," the company said.

"Elasticity of the cloud allows us to add thousands of virtual servers and petabytes of storage within minutes, making such an expansion possible."

Cloudy journey

The migration to the cloud was not as easy as cloud vendors' PR might suggest, warned the company. "The truth is that moving to the cloud was a lot of hard work, and we had to make a number of difficult choices along the way," said Netflix, explaining why it took the company seven years to complete the migration.

"Arguably, the easiest way to move to the cloud is to forklift all of the systems, unchanged, out of the data centre and drop them in AWS. But in doing so, you end up moving all the problems and limitations of the data centre along with it."

To avoid this problem, Netflix took a "cloud-native approach", meaning that it needed to rebuild all its technology, including optimising its data model with NoSQL databases, and changing the way it operates to fit effectively into a cloud-only environment.

Furthermore, Netflix adopted the DevOps and agile development approach to the migration, moving away from hardware provisioning cycles and central software release coordination to a more continuous model of development with engineers equipped with self-service tools.

"Many new systems had to be built, and new skills learned. It took time and effort to transform Netflix into a cloud-native company, but it put us in a much better position to continue to grow and become a global TV network," said Netflix.

In many ways Netflix could be a solid role model for cloud migrations. Many digital companies are effectively "born in the cloud", but Netflix started life as a DVD rental company.

Adopting the cloud was therefore a more arduous task, yet it serves as a good example of how a carefully considered approach can deliver the changes and results promised by the cloud.