Agile development methods avert costly failures

Department for Work and Pensions plans to base all future software development on agile software project management frameworks

Part of a wider IT cost transparency drive would typically involve an analysis of the money spent on software licensing costs, but it should also pay close attention to how much individual government departments pay for bespoke application development, either to their own in-house programmers or external software houses.

The litany of high-profile failed software development projects – think NHS National Programme for IT as one example – suggests a large amount of money in both public and private sector organisations is often wasted on software that is delivered too late, over budget or not fit for purpose by the time it arrives in users’ hands.

Statistics compiled by the Standish Group in its annual Chaos summary suggest that in 2009, nearly one in four (24 per cent) IT projects within public and private sector organisations across the globe were considered failures, either having been cancelled before they were completed or delivered but never used.

Gartner figures from 2008 also suggest a 20 per cent failure rate on average, with 25 per cent of those applications falling down on functionality issues, 15 per cent because of cost variations, 20 per cent due to cancellation and 18 per cent because they were delivered too late to be of any use.

“This is not getting any better despite all the money being thrown at the problem,” says Peter DuPre, chief solutions architect at application life cycle management and testing firm Micro Focus. “We would hope that software project failures turn around – we don’t have all the answers, but we do have ideas.”

One of those ideas is for more public sector organisations to switch away from traditional software development management structures – such as waterfall or cascade – towards an agile software development methodology now used in 70 per cent of Micro Focus’ projects, both internally and for its clients.

The Department for Work and Pensions (DWP) is a recent convert, pledging that all its future software development will be based on agile software project management frameworks.

Steve Dover, the department’s programme delivery director, believes unnecessary expenditure can be minimised by better collaboration between software teams working on the project and constant engagement with the end users for whom the application is intended, to ensure the software matches both initial and evolving user requirements.

“In most of the businesses I have dealt with, IT is a fantastic enabler which can reduce operational cost, make the business more efficient and provide better customer service,” says Dover.

“But one of the things I have really puzzled about in the past 24 years is why can we not specify what we want properly so the IT guys can build something that end users really want?”

Micro Focus’s DuPre believes the reasons why both public and private sector organisations waste money on procuring new applications from third parties that fail to match their expectations is clear – it is based on his own estimations that about 80 per cent of organisations are consistently disappointed with the outsourcers they commission to build applications on their behalf.

“The principal source of failure is in the initial requirement phase and the interaction with the provider,” he says. “It is the lack of being able to simulate what users are looking for, play it back to them and validate it before they actually start spending money on developing the software. Prototyping and interactive simulation are not new concepts, and are becoming much easier to do and more logical within the software development life cycle.”