Getting decisions made - thinking about Agile and open data when implementing analytics projects

Birst's David Gray discusses how business intelligence can be agile, and how it can benefit the UK's public sector

Business intelligence (BI) has been around for over 30 years, evolving from the decision support systems of the 1960s. It undoubtedly can and has provided huge benefits for private- and public-sector organisations, but these kinds of projects have often been limited to large centralised teams that have had the technology skills, resources and the budgets to implement traditional BI. This has meant that smaller organisations or departments have either been left behind, or ignored, often leading to a great deal of frustration.

The advent of cloud computing has made it easier and quicker to deploy BI. However, the challenge has been to get over some of the traditional thinking that organisations have still had around BI. In the UK public sector, this mindset is changing. The growth of the G-Cloud programme has made it easier for public-sector bodies to evaluate and purchase cloud solutions. Alongside this, there is increasing interest in making access to data for analysis more open as well.

The UK leads the world in open data initiatives, according to the World Wide Web Foundation's Open Data Barometer. The work of the UK's Open Data Institute has led the way on how data created as part of services for citizens can be made available for them to view and interact with. The organisation's roadmap provides a guide to how this strategy will work in practice.

For BI projects in the public sector, access to data has to be safe and secure even as greater availability of that information is considered.

However, this may not be best rolled out in one "big bang" project. The reason for this is simple: it's harder to think about the needs of numerous distinct audiences at one time, compared to meeting the needs of one group. Instead, it's important to look at this as a staged approach, where internal use of analytics is delivered first. Following this, it's possible to expand the availability of data further - either through allowing trusted third parties to get access to data and dashboards, or by making that information available to the wider population.

Agile software development was originally created to run projects that were designed to keep pace with changing organisation or business requirements over time. Rather than getting affected by a change in requirements mid-way through a project, the Agile approach looks to meet the needs of each team or set of stakeholders as things change.

This is a good mindset to approach open data and BI projects with, as this allows the BI implementation team to concentrate on meeting the needs of the internal team first then looking to expand outwards to other audiences over time. By treating the internal requirements for data and analytics as a quick win, it's possible to deliver better data for decision-making faster. Over time, this can then be expanded further.

Taking an "agile" approach to BI is a great idea in theory - turning it into practice involves maintaining control over the central data sources while also looking ahead at how problems or requirements are changing too. Without this control, it will lead to a more unstructured approach that won't provide as much benefit back to the organisation. In order to take an agile approach forward, the BI and business teams have to work closely together to make sure that the data sources and the questions asked continue to be relevant to the public body.

Thinking about external audiences and use of data should also be considered as part of the long-term roadmap for any BI or analytics project. Areas like compliance and security are important when the data is going be used internally, but making information available to a wider citizen base adds more challenges, particularly around anonymisation and security.

For UK public-sector organisations, BI offers new opportunities to improve efficiency and perhaps more importantly it can demonstrate where great work is already taking place.

David Gray is vice president international at Birst