Runescape: A big data case study

Stuart Sumner dons his helmet and shield and explores the IT infrastructure that serves the 200 million beast-slaying denizens of RuneScape

There is money to be made in killing dragons, and not just the virtual kind. One of the best-known online games, World of Warcraft, charges its 10 million users around £9 per month for the privilege of playing. Activision Blizzard, the game’s publisher and developer, recently announced third quarter revenue of $751m (£471m).

However, some games fare very well without employing the subscription model. In fact, the world’s most popular MMORPG (massively multiplayer online role-playing game), with over 200 million registered accounts, is RuneScape.

The player-base is split across multiple servers, since no affordable system could possibly support that many players logging on simultaneously, let alone the fact that the game’s world itself would become so crowded it would make the London Underground in rush hour look empty.

“An individual game server application supports 2,000 players at once,” says Remo Gettini, chief technology officer at Jagex, the game’s developers. “We can have a number of these on a physical server, usually somewhere between two and 11, depending on the server hardware. The exact number of concurrent players scales with the number of physical servers we have.

“We tend to sit at around 50-60 per cent of our capacity, with room to turn on more servers if we need to, but in a normal day at peak we have hundreds of thousands of concurrent players. This number has remained fairly stable over the past few years, although the number of game servers a physical server box can support has increased as we have obtained more multiprocessor machines, so we have consolidated a bit.”

Uninterrupted uptime is incredibly important for a MMORPG. Unplanned server outages quickly result in frustrated players, even when they’re not paying for the service (though RuneScape offers some premium features for paying customers).

Although like most firms Jagex is cagey about its security and disaster recovery planning, it tells Computing that the game servers run Linux, and the firm uses information infrastructure firm EMC for its storage needs. All important data - including that relating to players and their accounts - is replicated to its disaster recovery site, as well as the developers’ office for tape backup.

Gettini explains that avoiding complexity is part of the answer the firm has found to battling security issues.

“We set up our servers to be as basic as possible and run only what is required. We do have a contractor party who carries out penetration testing on a regular basis. Most of the attacks are DoS- [denial of service-] based, which are mitigated by bandwidth and our IPS devices.”

While this policy has proved to be fairly successful up to now, he believes the government is not doing enough to protect UK IT businesses from cyber crime.

“I don’t think the government will ever do enough to promote or protect IT businesses, primarily because securing your assets, whether they be electronic, intellectual or physical, is typically perceived as the responsibility of the owner. Just like you have to lock your car when parked, or lock and alarm your home when you leave, you are also expected to secure your own IT infrastructure applications and information. Clearly education is part of any government’s mandate but this is a complex issue and perhaps even beyond their grasp anyway.”

Runescape: A big data case study

Stuart Sumner dons his helmet and shield and explores the IT infrastructure that serves the 200 million beast-slaying denizens of RuneScape

The game itself runs on an internet browser, which makes it playable on a vast array of systems.

“The game is almost entirely written in Java,” says Gettini. “The server is all Java, except for about 20 lines of C. All of our development is done in-house. Prior to the recent graphics overhaul [dubbed RuneTek 5, Jagex’s own proprietary game engine], the whole client was Java, with pure software rasterisation, but now the client has some native code for the graphics. This is mostly C++, which is used to interact with DirectX and OpenGL, with a bit of objective C for interacting with some Mac OS libraries.

“There is also some use of intrinsics to do some heavily optimised software rasterisation, for people who have strange graphics drivers or who find they get better performance from this, which tends to be the case on some netbooks.”

Running virtual worlds that allow millions of people to run around, killing dragons, interacting and trading with one another creates a vast torrent of data, which Jagex captures and mines to better understand how its users play the game, and thus how their experiences could be improved.

“We feed about 2TB of data from RuneScape into a Hadoop cluster every day,” says Chris Smith, head of business intelligence at Jagex.

“This data includes player movements, interactions with the game, trading and combat. We have a dedicated Game Intelligence team of analysts for RuneScape who use Hive as a way to interrogate our Hadoop data. The game analysts sit and work closely with developers and designers to create the best possible game experience. A great example of this in action is on our Evolution of Combat beta [a reworking of the game’s combat mechanics currently in trial phase]. We have an analyst supporting this project full time, putting hard numbers behind the ideas and improvements to make the biggest leaps forward in the shortest time.”

Smith extolls the virtues of SQL as his data management tool of choice.

“While there are many tools that can quickly visualise data, it often comes full-circle to the flexibility of simple SQL. This fits with our use of relational databases and Hadoop Hive, and we’ve found Excel to be an extremely accessible standard for consuming data. We also use complementary technologies like R and Pig for data mining, and Tableau for fast visualisation.”

While much has been said of big data in the press, the actual examples of use cases among traditional businesses is still tiny. Meanwhile Jagex, one of the firms at the bleeding edge of big data exploitation, is examining the information that comes from people sitting at their computers shooting arrows at goblins and baking virtual cakes.