How bet365 shifted to containers with Kubernetes

James Nightingale and Alan Reed explain how bet365 shifted into containerisation with Kubernetes

Gaming company bet365 wouldn't have achieved the success it has without being technically progressive.

As the company has grown, we've had to consistently experiment with new technologies and pioneer new ways of working in order to meet the demands of our customers.

That's not to say that we're not pragmatic in our technology choices. For instance, we don't jump onto every trend just because it is enjoying a period of hype.

In fact, the more hyped a technology is, the more pragmatic and questioning we become about it. The problem with following hype is that you never know whether the technology will survive or if you'll find yourself halfway through an integration, only to see the community around it dissipate.

And so it was with Kubernetes and containers.

The idea that Kubernetes might make a good fit for bet365 was first mooted by the Infrastructure team in 2016. They had been following the technology's progress and felt it could solve several fledgling issues that were making the job of Infrastructure more complex to achieve.

Infrastructure's challenge

Ideally, the aim of Infrastructure is a straightforward series of tasks that ensure it can quickly spin-up environments for Sports Development to deploy its applications. However, as the company and the platform has grown, the job has become more challenging.

The bet365 system is large, complex and has many moving parts. Spinning up classic infrastructure takes time. There's lots of configuration involved to ensure applications run smoothly, can integrate with the rest of the platform and be ported around the globe.

But there's also a growing grey area around who has responsibility for the config. There are elements of configuration and tuning that are deployed across the stack, which would traditionally be the remit of the Infrastructure team, that need to be tuned by Sports Development.

The result is two teams touching the same configuration item. This added complexity increases the potential for error. It was this potential for error-inducing conflicts that led Infrastructure to investigate ways of working that would ensure both teams could get on with their jobs

The Infrastructure team was aware of modern approaches that were showing a lot of promise. They liked the idea of immutable infrastructure and started exploring options with bet365's R&D department.

Initially, we looked at reinventing our approach around virtualisation. We explored Openstack and DIY virtualisation (unikernels) but they didn't fit well with the business. As we worked through the options, we found containerisation held the greatest potential and began to gravitate towards Kubernetes.

We decided that we'd build the platform ourselves. We'd have to spend time learning how to do it but, ultimately, it would give us greater agility once we were up and running. As expected, it took some time to develop a working model for the platform, but not just because of the technical complexity.

In truth, we got a little lost in the art of the possible: there were endless discussions around what components we were going to need and how they'd be implemented. With Kubernetes, there are so many projects around the ecosystem that it's easy to get carried away.

We realised that we were trying to build the perfect solution before mastering the basics. In the end, we decided we'd build the simplest platform possible, prove the model and then develop it from there.

bet365's R&D department created a working model and Infrastructure undertook the proof of concept using a trader tool. It wasn't a full production workload, but it gave the team a feel for how well the platform performed.

The proof of concept demonstrated that Kubernetes could work with bet365's tools and that containerisation was compatible with the ethos of Infrastructure.

However, although a good learning exercise, we needed a test case that was more compatible with Sports Development. The problem was, Sports Development was not convinced by containers.

[Please turn to page two]

How bet365 shifted to containers with Kubernetes

James Nightingale and Alan Reed explain how bet365 shifted into containerisation with Kubernetes

Sport Development's challenge

While Infrastructure was working on their Kubernetes platform, Sports Development was busy trying to solve its own technical challenges. The most pressing is always the need to get code out faster. We work to fixed deadlines and are always under pressure to get fixes and new releases out as quickly as possible, or risk missing a major sporting event.

Over the software development life cycle the release passes through a range of departments, all of whom handle the code and make changes. Testing is ongoing and each time a change is made, you need a tailored set of instructions bespoke to that particular instance. What we really needed was a universal way of doing things - a universal language that everyone, at every level, understood.

Like the infrastructure team, Sports Development was investigating several different approaches that promised a solution to their challenges.

We looked at some automation products for build and release. However, we found that while deployments could be measured in minutes, the number of checks and balances required to ensure the code had been deployed successfully took too long - so long that it was still taking too long between production and release.

The goal was to ensure the facilitation of the code was as brief as possible, to everywhere at the blink of an eye. But with the flexibility to turn it on, turn it off, roll it back and balance it across geographically dispersed environments we needed a fast deployment model that gave us consistency, availability, and abstraction - but not the worry.

Initial scepticism

The Sports Development team was sceptical about containers because of the amount of code that would be transferred. We were used to a programme of delta releases, which ensured that only essential changes were made and the risk of technical issues were reduced.

Kubernetes looked good but there's a risk that you're just adding layers of abstraction and when things go wrong, you find yourself trying to cut through that abstraction. We understood containers conceptually and could see the benefits but didn't want all the noise that came with it.

In addition, the concept of moving large amounts of code each time we released didn't sit well. We felt it would be too slow. Then we saw Infrastructure's demo and were blown away by the speed at which applications could be deployed via containerisation.

So began the search for a candidate that was malleable enough, atomic enough and portable enough to demonstrate that we could make Kubernetes work. But that also had limited impact on the customer and the infrastructure - just in case.

The authentication module for Italian Horse Racing was the perfect fit. It's localised to Italy and horse racing, and wouldn't have too big an impact if it didn't work. But if it did work, we could use it for all streaming and, if we could use it for all streaming, we could be looking at a much wider variety of components. It would be a first tentative step towards a highly repeatable model.

Two challenges. One solution

Looking back, it is now clear to both teams that they were trying to solve the same problem, but coming at it from our own perspective and gravitating towards technologies in our own fields.

Infrastructure wanted to remove complexity and increase the speed of spinning-up platform architecture, while Development wanted to speed-up releases through a highly malleable deployment mechanism. Kubernetes has helped solve both demands.

We didn't realise it at the time, but Kubernetes is a natural next step for us. The move to Linux and the ability to transition onto languages like Golang has made containerisation possible.

So far, we've created a simple set of production clusters that have proved that Kubernetes is the right approach. We know that it can run production applications and that it is the right set of technologies to advance the bet365 sports betting platform.

We envisage that in the same way that Erlang transformed the core systems, now Kubernetes and Golang may be the tools that transform the deployment of our customer facing applications.

James Nightingale is Principal Infrastructure Architect and Alan Reed is Head of Sports Development, Hillside Technology, at bet365's technology business

Delta is a new market intelligence service from Computing to help CIOs and other IT decision makers make smarter purchasing decisions - decisions informed by the knowledge and experience of other CIOs and IT decision makers.

Delta is free from vendor sponsorship or influence of any kind, and is guided by a steering committee of well-known CIOs, such as Charles Ewen, Christina Scott, Steve Capper and Laura Meyer.

Ten crucial technology areas are already covered at launch, with more data appearing and more areas being covered every week. Sign-up here for your free trial of the Computing Delta website.