Despite all the fake news, ODI chief Sir Nigel Shadbolt remains 'resolutely optimistic about the emancipating power of good data'

The UK needs to treat data as infrastructure in the same way as roads and the power grid

On the face of it, these are grim times for those who set store by empirical evidence. With propagandists of all stripes moving to take advantage of the displacement of traditional sources of information sources by social media, verifiable facts have seemingly fallen out of favour, along with those who labour to establish them.

However, chairman of the Open Data Institute (ODI) Sir Nigel Shadbolt sounded confident that the situation will right itself when Computing spoke to him recently.

"I remain resolutely optimistic about the emancipating power of good data," he said. "Whatever your political allegiance I can't believe you'd not rather be in possession of valid data."

This argument is supported by the fact that the demand for data expertise has never been higher, he said.

"Wherever you look, businesses can't get enough of people who can wrangle data, transform it, present it, ingest it, clean it. People worry about the jobs it'll take away but it's a whole new class of jobs that are being created."

This is the data-driven world that the ODI, which recently marked its fifth birthday, was set up to facilitate. The brainchild of co-founders Shadbolt and Sir Tim Berners Lee, it is an independent not-for-profit company that focuses on incubating startups that seek to harness open data, as well as providing training, encouraging "data literacy" and promoting standards.

As well as the prevailing mistrust of data and experts, the ODI has had to deal with a few misunderstandings about its mission, one being the misconception that it believes that all data should be open; in fact, all information ideally lies somewhere on a spectrum between open and closed. So a bus timetable should be open, while a company sales report should not be. Another common misconception is that making datasets open will leave them open to abuse.

"People said 'if we open this stuff [transport timetables] up there'll be all these timetables out there that'll be full of bad data'," said Shadbolt.

"I said 'if that happens they won't work so people won't use them'. There is a market for accuracy and veracity. That's not to say people won't seek to abuse of subvert or variously misrepresent things but I'd still always come back to the value of high quality data."

Building a data infrastructure

The central importance of data to the functioning of the modern state means that key information sources should be treated as a public good: cleaned up, curated, digitised and made available via an API over the internet, argues Shadbolt. Examples of data infrastructure include geospatial data, building regulations, NHS procedures and processes, locations and opening times of hospitals, as well as the data maintained by Companies House.

"A well curated list of legally constituted companies is part of our infrastructure, all of the timetables for public transport are part of our infrastructure, they should be guaranteed by the state in the same way as roads, clean water and clean air," Shadbolt said.

"That doesn't just happen, it needs to be engineered and developed. We have some examples like Companies House data and Ordnance Survey data but there's so much to do there."

While the ODI has scored some notable successes in persuading government departments such as Food and Rural Affairs (Defra) and Transport to make available many of their datasets, Shadbolt expressed frustration at the rate of progress.

We don't enjoy a massive level of resource like GDS, but at least we don't get struck down halfway through the process.

"People have talked about this and got quite excited about it and I'm surprised there hasn't been more of a national effort of will," he said.

"We had thought that GDS [Government Digital Services] would be where this might happen, but for various reasons we feel that there's still work that needs to be done. I think you'll see the ODI playing a bigger role in that.

"We're not part of government and we don't enjoy a massive level of resource like GDS, but at least we don't get struck down halfway through the process. At the GDS the issue is who's responsible? Who's the guiding mind? Which department actually owns the problem? How do you get a consistent adoption of open data policy across government? It's really not an easy thing to solve."

Net neutrality

A big part to promoting innovation around data is ensuring equitable access to information, another ODI aim seemingly at odds with current events. This goal is threatened both by attacks on the principles of net neutrality by the US government at the behest of large carriers like Comcast and AT&T and also by the monopolisation of power by governments and large corporations.

"The modern corporate infrastructure has been centralised," said Shadbolt. "The deal between consumers and corporations is skewed, lots of value is extracted and what's the deal back in terms of control? It's not just the technology but also at the level of content providers."

The deal between consumers and corporations is skewed, lots of value is extracted and what's the deal back in terms of control?

The ODI is supportive of technological efforts to put control of data back into the hands of individuals and to encourage startups looking at technical solutions to the centralisation problem, but ultimately Shadbolt sees this as a political and cultural matter that society at large must resolve.

"The issue of a neutral internet is about legal and social norms, about equity, high political principles, the balance between citizen and state, between governments and businesses, and businesses and individuals. There are deep issues about fairness. We need to adjust those constants."