How CTO John Linwood is leading Wood Mackenzie's charge to 'go digital'

When Linwood joined the firm, his brief wasn't just technology, but helping the organisation to re-architect itself to 'go digital'

For the media and other publishers of information, the internet age can only be described as "challenging". Newspapers, for example, have struggled to adapt to a medium in which readers resolutely refuse to pay for the product, notwithstanding the success that a couple of exceptions have made with their subscription models.

Even the market for high-end business information has been challenged, on the one hand by the availability of vast amounts of free information and, on the other, by clients demanding deeper and ever-more sophisticated information and analytical tools.

It is the latter that John Linwood, chief technology officer (CTO) at Edinburgh-based oil and gas information company Wood Mackenzie, was challenged to address when he joined the organisation in June 2014, not too long after a well-documented departure from the BBC.

As a company in the business of business information, it doesn't just have the usual challenges of a publisher in the internet age. Having been around since 1973, it's also carrying with it the baggage of legacy systems.

For Linwood, therefore, the strategy is to re-architect the organisation's systems, which will provide the foundation for more profound change throughout the business, all the way up to the products and services on offer to clients.

"We have a big re-architecture project going on at the moment. Here's the challenge: like everyone, we have many silos of technology and, over time, we've built a lot of stuff, we've acquired companies and systems, and it's very hard to get them all to talk to each other, keep them up-to-date.

"So we have a very large project at the moment where we're moving to a completely services-based architecture. What we're doing is, essentially, re-architecting as we deliver projects. Each new project comes along and delivers the components that it needs in the underlying architecture to be successful," says Linwood.

As the new infrastructure is rolled out, the organisation is able to deliver systems more quickly, as well as providing tools and technology to the front line.

SOA on the way

Linwood expects that as the service-oriented architecture (SOA) is rolled out, analysts will be able to directly "play" with the underlying (and currently siloed) data in order to glean more insight, write more in-depth reports and provide even sharper analysis - and more data - to clients.

"I'm very keen to allow the analysts themselves to start using tools like Python and R to create both customer offerings and tools that can help them.

"One way to do that is to give them all of the services underneath so that they don't have to reinvent the wheel entirely. The idea is very much to have a set of services. They then just have to write the module they are interested in, and they can call whatever services they need underneath," says Linwood.

He continues: "I want to enable the data analysts, the people in the business who have research backgrounds, statistical or deep analytical backgrounds to be able to build services on top of that.

"If you think about it, they have already done that in Excel and Access. Find me an analyst who doesn't build thousands of Excel spreadsheets or Access databases. So the idea is to really empower them to go to the next level and start to create more and more value.

"We've had some good examples of that already. We've had teams, particularly in our Houston office, where we have an innovation centre, who have started to build customer-facing components out of Python.

How CTO John Linwood is leading Wood Mackenzie's charge to 'go digital'

When Linwood joined the firm, his brief wasn't just technology, but helping the organisation to re-architect itself to 'go digital'

As for the analysts, though, are they really up to this challenge? "Pretty much all of them have used Visual Basic in some form, so they are used to developing stuff. But they may not be used to using those technologies [R and Python]. Part of it will be about providing training, and also providing them with an environment that makes it easy and ‘safe' for them to do that.

"Provided we get our services layers right, it means that whatever they do will be ‘safe' in that it won't be directly accessing the data; it's going to be using a service technology that adheres to all the standards of security and resilience, but at the same time enables them to get the job they want to do done," says Linwood.

But the use of R and Python to delve deeper into corporate data isn't solely limited to analysts internally - the aim is also to deliver new tools to customers, too, so that they can dig deeper into Wood Mackenzie's data, but it is still early days.

"We've designed our high-level architecture framework, we've got two projects underway starting to deliver components of that. We have done some training in terms of bring some staff up to speed in using these tools. I would say we're in the first semester of doing it. We've launched a couple of products out of the Houston office but it's still early days," says Linwood.

"Having said that, we have 500 analysts who are used to creating value through Excel and Access databases, so it's not too different to what they were doing before, it's just a different toolset that they need to master."

Historically, he adds, Wood Mackenzie has followed a waterfall-based methodology, but for the all-new projects has shifted to agile.

"It's a journey. The new projects that we're running are agile, so the e-commerce project, for example, was agile; the mobile project, which is now producing its second-generation of products, is agile. But then a lot of our historical projects have been waterfall," he says.

Database decisions

The current re-architecting, though, is merely just a first phase intended to free data for deeper and wider use by analysts and the business generally. A longer-term consideration will be the very form in which data is kept across the business - one in which Linwood and the IT team are actively questioning the need and utility of relational databases.

"The re-architecture is, essentially, building a completely new framework that, first of all, ties together all of the back ends of our systems so that you have a single view of the data, and can share components across all of our systems. None of this is rocket science, it's basic common sense," he says. "Over a period of time - probably two to three years - we'll replace all of our back-end components with new services."

Many of the new services have, so far, been enabled by wrapping them around the existing back-end infrastructure, which over the next few years will be replaced. And that may mean not just a shift to new database technologies, but also a complete re-appraisal in the way in which data is stored and distributed throughout the organisation.

How CTO John Linwood is leading Wood Mackenzie's charge to 'go digital'

When Linwood joined the firm, his brief wasn't just technology, but helping the organisation to re-architect itself to 'go digital'

"Relational databases have been a bit of a cludge - they try to be all things to everyone. So if you are doing large amounts of bulk processing, you put it in a relational database. If you're doing querying, you put it in a relational database.

"But the bottom line is that they're not great for many of the things they're currently used for... One of the things we're looking at is abstracting the data layer so that we could hold the same data in many different formats, depending on what we're doing with that data," he says.

Hence, the database is less important than the components that can be wrapped around the data to create new applications and services, and to enable Wood Mackenzie to be more responsive.

"By creating components that wrap this [data], we can wrap our existing data services, over time, and replace them with smart decisions around where the data's going to be located, how the data is going to be stored for that particular task. For that, we're looking at Cassandra, Hadoop, and also Oracle and Microsoft SQL Server, because they'll still be around for a very long time.

"There will always be a place for relational databases," he adds, "but we're looking at ways in which you can store the data more effectively."

This re-examination of the database and the place of data within the organisation hasn't just been stimulated by competition and the need to be more nimble, but also by technology developments such as cloud, he adds.

"We're really interested in cloud and how we can distribute our data more effectively around the world. Our current architecture is based on big, centralised databases sitting in a data centre - systems sitting on top of those databases and, essentially, using the web to distribute that with content delivery networks (CDNs) on the front end, so we use Akamai there. The problem with that is that everything is coming back to the one place. So that becomes a bottleneck," says Linwood.

He continues: "In the new architecture, the idea is that the data is distributed, but in essence that's handled by the data layer. You don't need to think about that. As you're building your apps on top of that, you just know that you call that service and it's for that service under the covers to decide how to best store that data and where to store that data.

"In essence, data will appear in multiple places around the world according to the most appropriate way to deliver the data for that particular request or service," says Linwood. Or, at least, that is the long-term plan. For an organisation like Wood Mackenzie, which uses data for analysis, a NoSQL database might make sense, he continues. "If you're doing something analytical, like MapReduce, or you're analysing a huge amount of data, you want to be able to get through it really quickly, then you don't want the overhead of a relational database," he adds.

How CTO John Linwood is leading Wood Mackenzie's charge to 'go digital'

When Linwood joined the firm, his brief wasn't just technology, but helping the organisation to re-architect itself to 'go digital'

Ultimately, Linwood foresees data increasingly being kept in its native format, rather than sliced and diced in order to make it fit into a corporate relational database, and that data being extracted in its raw form and manipulated according to the needs of analysts, clients and the business.

Universally challenged

And the challenges - common to all organisations, not just Wood Mackenzie, don't stop there: attracting IT talent remains a big issue, says Linwood, with the best mobile developers, for example, simply wanting to work on the coolest projects before moving on to the next exciting challenge.

Speed of execution, too, is also a challenge in a market that's changing faster than ever. "There's huge pressure to build new products and services that customers are demanding," says Linwood.

"The other really interesting one for me is, how can we really leverage a lot of the technology innovation going on in the market place because of the speed of innovation is high, so you can lock onto something, then months later it's old news!

"The risk as a technologist is that you're looking at the next interesting thing that's coming along and it's important to stay focused, to deliver and realise the benefits of what we're working on. So there's a tension all the time between the new technology and the need to get existing projects done and delivering value for the business," says Linwood.

Indeed, the business of business information may be more challenging than ever, but they're challenges that pretty much every business is going to be facing, sooner or later.