Agile methodology: Can you ever have too many agile projects?

Agile is used by firms to increase the speed of a project, but research suggests that too much of agile can actually slow things down

CEOs are demanding organisations move faster, according to best practice insight and technology company CEB, and of course, one of the ways to move faster is through technology.

And so CEOs are increasingly looking to CIOs to move things along.

Take Patrick Seeber, CIO at chemicals and sustainable technologies firm Johnson Matthey. He told Computing that he was brought into the organisation to look at new ways of working with IT.

"We're transitioning from a ‘big small company' to a ‘small big company', and we want to hold on to the things that have made us successful, but also look at things we could do better in terms of scale, efficiencies and certainly speed, because if we don't organise ourselves or support the business, the business will find a way of doing their own thing without [IT]," he says.

The company has a total of about 150 smaller businesses that it is combining - but it doesn't want to lose the agility and flexibility usually associated with smaller businesses.

"So we developed a new strategy, with a new operating model for IT, which changes the way we are set up... and the challenge for me is getting those adaptive procedures and ways of working built into our internal processes that we have in our organisation," says Seeber.

Introducing agile

One of the methodologies that many organisations are reverting to in order to achieve that added speed is agile.

It's an alternative to traditional project management, and is usually used in software development. Essentially, it requires collaboration between self-organising and cross-functional teams, in order to rejig requirements and solutions in shorter time-frames.

One of the main approaches in implementing agile seems to be a "two-track" or "two-speed" approach to IT. That is, one part of the IT department working in the way that they always have, and the other working in a faster, more flexible way - but on a different kind of project.

As Reckitt Benckiser CIO Darrell Stein explains, this is an area where the organisation can afford to take a few more risks.

"This is where you can try out newer technology and if you make a mistake, you can take a 'test and learn' approach where you're testing something and if it doesn't work you can pull it out - you can't do that with a global ERP system, which is affecting the whole world [of your business] in one go, and if you make a mistake it'd be a big issue for you," he suggests.

Cathal McGloin, VP of mobile platforms at Red Hat, describes the concept of "two-track" as the following:

"The whole idea is keeping existing systems and skills running, but also introducing a new track with new skills, new tools and new ways of doing things that are more agile. You keep them alongside each other, rather than trying to change things altogether," he says.

He advises firms to try the method out on a simpler mobile development project, rather than starting with a large-scale complicated project.

Computing has heard of several agile success stories: Tesco took the approach when it came to rebuilding its data warehouse, while ITV used agile to move from tapes to digital.

Can there ever be too much agile?

According to CEB's 2015 State of Agile Survey, the answer is yes, there can.

"We see agile as speed by carve-out; so making one part of IT faster, but what we actually found is that it's a valley of death - if you do a little bit of agile, it gets a little bit faster, if you do more, it stresses the organisation so it slows down," CEB managing director Andrew Horne says.

[Click here for an enlarged image]

If a company uses agile for up to 15 per cent , or one in six or seven projects, they can get faster, but then there's a trough where if they try and do more than that. There are so many delays that it actually begins to slow things down. For the small number of organisations that get out of this, the speed picks up again, but Horne says that the majority of businesses are stuck in the middle stage.

"It's like if you buy a car and put a much bigger engine in it, then it still can't go that fast because everything else is going at the same speed, and so you have to upgrade everything at the same time," Horne explains.

"So we're not saying that agile is not the thing to do, and a lot of people have had success with it - it's more that it's not enough. If you really want to go fast you need to do agile and [increase] the speed of everything else around it so the organisation can move faster," he adds.

But why is there a ‘trough' stage?

"We found that up to 80 per cent of the delays that occur in IT happen before you officially start the development process; deciding what you're going to build, getting the budget lined up, selecting and negotiating with a vendor, getting architecture, compliance and security in place.... All of the managing before you start something like agile creates a great deal of delay," Horne says.

The delays don't end once the project has begun.

"Once you build something there are still delays. We heard from one big bank who were very proud that they had a special team that created a customer-facing app in a week, but it took nine months to get it live because of all of the other stuff that had to happen," Horne adds.

Much of this is to do with the attitude and goals of the business.

"There is a limit you can reach where you are effectively running one giant agile project, with the individual projects becoming sprint tasks. This can work really well in an R&D or product development type environment, where you may expect a few projects to slip or fail, but it's less effective in a client service delivery team where the expectations are different," says Peter Kinder, CTO of Wax Digital.

Rorie Devine, former CTO of Betfair, and now CEO of TeamCXO.com, believes that if the agile projects aren't managed at the enterprise level, then they are bound to slow things down.

"When you create independent, self-organising, teams they are independent. You also need to create an enterprise-wide framework for them to operate within in a consistent, coherent way. You need to manage dependencies and avoid duplication across teams. You are also normally trading off agility against predictability so more agility can mean less predictability for an organisation," he says.

But perhaps it's not just agile. LV= Insurance's fast-track development director Rod Willmott said that slowing down wasn't just an issue that is related to an increase in agile projects.

"Too many of any type of project eventually causes issues. Agile does not take away the need for prioritisation and projects still need technical and business focus," he says.

Willmott suggests that doing too many projects in parallel causes issues, and that one way of getting around that is to do more in series.

However, he warns users to be conscious of "follow-up phases", which can lead to a major backlog. So if you have several agile projects on at once, it may lead to some of the follow-ups being put off in order to focus on something new.

But while many believe that too many agile projects in any one business can slow things down, Jacob Tomaw, senior principal engineer at Orbitz, doesn't think this should ever have to be the case.

"If they are hampering the speed it is likely that they've not implemented the agile process in a way that actually decouples teams," he says.

How to ensure you don't get stuck in the ‘trough'

First and foremost, there must be business stakeholders who are fully engaged and who are made aware of exactly what agile involves. Other staff must be fully engaged too, as they will be part of a cross-functional team.

Reckitt Benckiser's Stein says that implementing a twin-track approach must be done with sensitivity. It can be hard for the "corporate" IT staff to see a new agile team working with cutting-edge technology.

And no matter what approach an organisation takes, there is an agile/waterfall boundary at some point in the organisation, which also has to be managed.

"Like any meeting of two dissimilar philosophies, there will be friction. The managers who straddle that boundary are critical and strongly affect whether the business succeeds with agile techniques," says Paco Hope, principal security evangelist at Cigital.

The higher the level in the organisation that agile is adopted, the bigger the chance of it succeeding, says Hope, but that doesn't mean that the buck has to stop with senior management. In fact, CEB's Horne encourages CIOs to push accountability down in the organisation without losing control.

"If it's someone junior or in the middle of IT who really should make a decision, they should just make it and move on," he suggests.

Much of this is dependent on whether the right controls and framework are in place for project managers to make their own decisions without going against what the IT department had agreed from the offset.

But while Horne suggests that it would be better for guidelines to be in place for project managers, he emphasises that processes should be flexible.

"There are many elements [of a project] such as risk assessments and getting architecture approval, and those are in there for a good reason, because prior to them being introduced [businesses] had chaos. But there is a risk that it slows the process down and you apply processes where you don't need them; where you apply too much process," he says.

He suggests that organisations should spend time figuring out when a shortcut is viable and when it is not.

"Maybe where a small project doesn't affect customer data and is experimental it may be okay to slash away some of the processes and get on with it, and if it is a global enterprise project, you want more of it," he says.

This may sound simple enough, but Horne says that companies aren't good at calculating which projects and processes should go through different teams.

One of the first things organisations should address is "handoffs".

"So how quickly can the architectural team get approval, how quickly can you get procurement from legal, and so on - all of these handoffs are creating delays," he says.

Finally, there has to be the right mindset and skillset in the organisation. This may require some recruitment; Reckitt Benckiser, for example, had to hire a completely new team to focus on agile.

Sometimes, simple name changes may help to adjust employees' mindsets.

"We're moving away from long programmes of work, in fact we've abolished the term ‘programme' - we don't have programmes anymore and we don't have programme managers," says Tony Scott, transformation director at engineering juggernaut Atkins.

"We don't sign any [project] off that's more than three months, and this is giving us a higher level of agility. At a lower level we're adopting scrum in our development so everything works in sprints. But we're also moving from project managers to product managers, which is another key change for us that aligns with our agile philosophy," Scott says.

CEB's Horne adds that you can tell when the mindset of IT has shifted - it's when staff really understand why agile (and other methodologies) are so important and how the company can benefit from faster throughput of projects. This will help them know when to take risks and when they shouldn't. It's also when they begin to think in terms of speed, cost and reliability, he says.

Computing says: As Horne says, agile can work successfully in many organisations. But relying on just one methodology can actually hamper success - particularly as organisations may find it difficult to manage several agile projects at once. It's important to look at several methodologies to increase speed. There are other things that Horne mentions that can apply to all projects, whether agile or not. If speed is what you're looking for, then you're better off improving existing processes, rather than increasing the number of agile projects you have on the go.