The upcoming Avengers movie has crashed ticketing sites: they should have learned the teamwork lesson

A superhero won't help when there are massive surges of demand

To say the upcoming superhero film Avengers: Endgame is highly anticipated would be an understatement. The marketing campaign included pre-order ticket sales, a new trailer and a massive batch of news stories to stoke interest -and it worked, setting a new record for pre-order sales at movie ticket site Fandango, beating the previous figures set by the likes of Aquaman and Avengers: Infinity War.

However, this surge in demand over a short period of time had a major impact on movie ticketing websites. Ticket buyers going through Fandango and AMC in the US, and UK customers purchasing through Odeon, had wait times of between thirty minutes and an hour, experienced website crashes, or were booted off the service before payment was taken.

With a launch of this magnitude, it can be extremely difficult for all the companies involved to plan for capacity across the entire supply chain. But it is possible.

Rather than rely on a single superhero to make a difference, teamwork is required to ensure all the pieces are in place before the traffic surge hits. Working together the team must ensure that services can scale up, work in real-time and be continuously available. They should look to improve customer experience through better processes, newer technology and expanded skills.

Testing the limits of scalability

Public cloud services have long been positioned as an option for scaling up and down rapidly in an elastic fashion. Adding capacity based on demand should be simple. Even if a product launch exceeds your wildest dreams and threatens to break the internet, you can still capture that market, or so the theory goes.

Unfortunately, simply trusting that your public cloud service provider will keep you out of trouble is not the answer. You may find that your initial plans for scaling up don't meet current budget or operations constraints, or worse, there are hidden hurdles that you hadn't planned for.

For example, your data architecture may meet day-to-day service levels without any problems, but a burst of activity or peak traffic may result in significant overheads. When you have a single node in charge of receiving all traffic requests and sending them out for processing, you have created a bottleneck that affects the ability to scale. Even with a limitless cloud service, that restriction could effectively halt future growth. Your ability to scale and meet spikes in demand is only as good as the slowest, most limited part of the architecture.

A masterless data architecture—where any node can take care of any transaction—is the solution. Further, a masterless data architecture can run in multiple places, all at the same time. Rather than being limited to a single data centre or cloud service, a masterless architecture can function across different cloud providers, thereby removing potential infrastructure scalability issues.

If you rely on a single cloud provider to handle surges in demand, then you also have to rely on their availability to scale when needed. Using a multi-cloud or hybrid cloud approach can help you keep some leverage in the conversation with your cloud service providers.

How real is real-time?

Another challenge for the backend infrastructure in response to a movie launch this size is how to respond to events and changes in real-time. First, defining what you mean by ‘real-time'—and what your suppliers mean by real-time—can help everyone understand what is required of your architecture, and determine how quickly you can respond.

For some technology providers, real-time can mean responding in milliseconds as transactions take place. For others, real-time can mean minutes between a set of transactions taking place and data being generated for analysis or alerts. The time involved depends on multiple variables—how quickly transactions can be analysed; whether analysis takes place in batches or per transaction; and how often alerts or reports are sent to people within IT or the business.

The most important criterion is how much time customers expect to wait during a transaction

And while the cost of running internal and cloud infrastructure can also be a factor, the most important criterion is how much time customers expect to wait during a transaction.

For major launches like the Avengers movie, today's data architectures need to accommodate millions of transactions in very short periods. More importantly, these architectures need to meet the needs of every customer, no matter where the customer may be.

Preparing for a major product launch involves a lot of teamwork and a modern data architecture. Estimating demand is a critical task that IT, business, and marketing teams must work on together. And, having a route to deal with your wildest levels of success is just as important. Cloud, hybrid IT and real-time application data architectures all have a part to play.

David Waugh is senior vice president of market development at DataStax

The AI and Machine Learning Awards are coming! In July this year, Computing will be recognising the best work in AI and machine learning across the UK. Do you have research or a project that you think deserves wider recognition? Enter the awards today - entry is free.