The top-ten tenets of software quality assurance, part four: Methods

A method should lead a project to the desired result without significant variation, with much of the thinking already been done for you, writes Mark Wilson

In system development, I believe a method is something you follow to convert inputs into outputs that consistently meet a stated requirement (the word "methodology" is properly what this article is - the study of a method).

You could also say that methods in IT projects are pre-defined ways of turning a project's requirements (new or changed) into delivered product. Many organisations follow project management and/or software development methods, but I have still found lots of occasions when there was:

A method [is] a crutch; something to lean on, that will lead to a desired result without significant variation, and where a lot of thinking has already been done for you

Agile, everybody?

These days it's hard to find anyone who is not using Agile methods (or claims to be doing do) but I've come across several places you can also find plenty who started using them but who welcomed back the best bits of the V-Model or ‘traditional' waterfall development with open arms.

Due to this established popularity, there is now a whole generation of developers who believe nothing good was ever developed before Agile, and who think the only way Agile methods fail is because "they weren't doing proper Agile".

Psychologically, Agile has a powerful mojo. What would you rather be: fleet of foot like a gazelle, or lumbering like an elephant? Of course, the outcome of the race between the tortoise and the hare never gets mentioned.

This popularity implies there must be some sort of consensus that methods - at least Agile ones - are a good thing. But it's not always the case.

Master Chefs

As a QA bod, it's always mystified me that people who wouldn't dream of setting out on a long car journey without directions, a map or a sat-nav, will quite happily wing their way through a complex IT system change without following a formally defined method.

A recipe is a good example of simple method. Would they attempt to make a complicated a dish they'd never made before without a recipe? Perhaps they'd just chuck it together and hope for the best, then order a takeaway if it turned out bad? I've seen people muck around with the recipe, chucking in strange, clashing ingredients in an attempt to impress, MasterChef-style.

You have to be serious about following a method - or it won't work

I can understand why this happens: there's a sort of confidence that things will work out, because that's the way they do things and have done for years. No point making a fuss about it. The company's making money and things get done, they feel. Software is getting released, and they regularly issue maintenance releases.

Yet in such places, it's almost certain that no-one is measuring the cost of this chaos: the repetition and the sheer waste. Probably, no-one's asking why they are constantly releasing software patches (if they did, they might find out that a good many are simply there to fix previous botched releases).

The psychology's quite important here, of course: people love being relied on and being indispensable. Writing it all down clearly so anyone can do it is actually a threat. And managers love fire-fighting. It's at places with these attitudes that "heroes" really thrive. Method is anathema.

It's really hard

Building or changing IT systems isn't easy; it's very complicated.

The ever-filling vat of jargon makes that complexity more obscure and even harder to understand (other industries don't seem to need this obscurantism - one of my oldest friends, an architect project manager, refers to even massive projects simply as "jobs"). To make it easier, you need a method. I like to view a method as a crutch; something to lean on, that will lead to a desired result without significant variation, and where a lot of thinking has already been done for you.

Methods typically consist of a specification of a process, and a set of procedures, standards and templates. Most people now choose some "flavour" of Agile. But which one do you choose? The trouble is, there seem to be SO many…

Many, many Methods

Waterfall, Iterative, Spiral…Agile, Scrum, DSDM, XP, Lean, Kanban, Crystal…

It looks like a question from Only Connect, doesn't it? What's next in this sequence?

In fact, that Wikipedia list above confuses models with software development techniques. And here's me calling them methods. Scrum isn't even a method, according to its co-creator, Ken Schwaber. It's a management practice.

He says: "Scrum is simple. It is the opposite of a big collection of interwoven mandatory components. Scrum is not a methodology." In fact, it's entirely possible to have PRINCE2 running the project, and Scrum (or another Agile practice) as the software management practice within that project. And PRINCE2 has mandatory products - thank goodness.

Playing at it

On scrum.org, Ken goes on to complain about "flaccid Scrum" (I think there may be the germ of a rugby song in there) and the fact that a lot of people think or say they're doing Scrum, just because they use the jargon and buzz-words and have meetings standing up and don't bother writing any documentation.

I agree: the adoption of Agile as a modus operandi has too often been an excuse for laziness and the ditching of thoughtful approaches; even the managers act like "school's out forever" when their company "goes Agile" when, in fact, if you are going to do Agile or Scrum, school is most certainly in, and in a very big way.

To develop fast, with quality, requires the latest development techniques and technologies, and the best development and integration environments. It also requires a revolution (for example, test-driven development or TDD) in the way testing is approached. Doing this means extensive, expensive training. Which is what so many organisations are NOT willing to provide. You have to be serious about following a method - or it won't work.

The reason methods work

Methods work because they introduce predictability. They've been written by people who've had a lot of experience doing this stuff before you; doing things wrong but also doing a lot more things right. They've done a lot of the heavy lifting for you. A method removes the guesswork: what happens next is what it says in the book.

I like to think of a method like cells in a spreadsheet that contain formulae: You input data here, and as if by magic, the complicated working behind the cell computes the result into a report; a report that gets value from all that data. The really hard part; deciding what data will be captured, and devising the formulae, has been done already. This is why you have to be so careful adapting methods: Missing out or ‘adapting' things like PRINCE2 and Agile methods means you're dispensing with those 'formulae' that someone else has spent a lot of time working out.

The best projects I worked on blended aspects of PRINCE2, Waterfall, and Agile

Rather than monkeying with a method that doesn't fit, find one that does, but then, stick with it. Use Agile when it meets the proper applicability criteria; if it doesn't fit, use something else (if, for example, there is a stable, known scope that has to be delivered all in one go, Agile is a daft way to do it).

Commonality

By all means consolidate ‘boards' and ‘stages' and ‘ceremonies', but never lose sight of the value of the outcome that the particular control has. PRINCE2 actually prohibits wholesale guesswork and says all of its seven principles must be followed. In any case, both PRINCE2 and Agile methods specify:

  1. Roles and responsibilities;
  2. A defined process model;
  3. A controlled start and close;
  4. Project delivery in chunks; and,
  5. Plan, risk, quality and change management.

At the risk of not being hip (I don't think I could ever be accused of that), I like the way that PRINCE2 furthermore emphasises the following:

  1. Continued business justification (see my number one Top Ten Tenet, Contract Review);
  2. Learn from experience;
  3. Manage by stages;
  4. Manage by exception; and,
  5. Focus on products.

Melting pot

I found that the best projects I worked on blended aspects of PRINCE2, Waterfall, and Agile. They were excellent in terms of timeliness (all on time), quality (zero defects passed to customer) and profitability - sometimes as much as 50 per cent gross margin. These weren't small affairs; they were in the magnitude of tens of millions of pounds.

The method was simply but clearly defined in a process specification document for the contract, and we produced a project initiation document, a test strategy document and a configuration management plan document.

In most of these projects, we used ‘features', which would be pulled from a prioritised requirements backlog, followed by a featured design document (all documents were properly reviewed and agreed), rapid development with input from the users, and early user-acceptance testing, with a product owner, as soon as a feature was ready.

There were several old-fashioned gates for the release-as-a-whole (the collection of features) for integrated non-functional testing. The features implemented the agreed architecture; they also delivered exactly what the business wanted and they performed well when they were implemented.

Methods work because they introduce predictability. They've been written by people who've had a lot of experience doing this stuff before you

Cautionary tales

Much too much

On one of the biggest civilian IT projects ever, my boss and I (okay, it was originally him but I improved it), came up with a list of work-products derived from the RUP (rational unified process) method and the huge, detailed contract we had to fulfil.

We then located some of the company's procedures and wrote some of our own, and devised some templates. It worked okay. But what started out as a product breakdown structure soon got over-thought when an army of process consultants were brought in.

It became a self-fulfilling monster that devoured time as it became more and more detailed and was so out of whack with common sense that deviations (both sanctioned and unsanctioned) became commoner than full compliance.

One day, while crossing the vast open-plan plain that was our building's fifth floor, I discovered what seemed to me like a long-lost tribe. Who were these poor wizened people, hidden behind dusty grey cabinets, eking out a miserable existence, bent over their primitive tools?

"What do you do?", I asked.

"We're working on the Method," came the reply. I never saw them again. They may well still be there.

You could be having fun

I was amazed to find that a digital channels department I worked for (an entire floor of people in their 20s and 30s) were delivering 12 releases a year of complex changes and new projects, but with no one project acting like any other.

What started out as a product breakdown structure soon got over-thought when an army of process consultants were brought in

Big visible charts were festooned with Post-IT notes that never seemed to move. Squads and huddles and tribes of deliriously happy young folk traipsed past my desk on their way to multiple meetings.

The guffawing and air-punching certainly was vibrant, but because there was absolutely no method and no process, it took me over a week just to find out what the current list of projects was.

Some things were tested, some things weren't. There were no customer journeys or business scenarios so no real user-acceptance testing could be done, and most of the small changes they worked on were fixes to fix previous fixes. But they were certainly having fun with their chaos.

QA AKA

I worked as test manager for a government department that paid unemployed claimants and also the training providers who helped them. While there was a widely-understood build, release and non-functional test process, it wasn't written down and the functional testing was based on copying the test documents previous releases had used.

One day, knee-deep in testing releases that were regressing with severity-one defects all over the place, I got an email telling me to fill in some inexplicable form or other. I was astounded to find it had come from the quality management team.

When I went up to find them I saw a shelf of ring-binders full of procedures and standards, but I couldn't find anyone who cared less about them, and the quality team could easily be ignored. All they wanted was someone to fill in their forms, and then everything in the garden would be rosy.

Mark Wilson is a Quality Assurance consultant with over 30 years' experience in quality management, quality assurance, test management and configuration management. He can be contacted by emailing [email protected]

Missed Part One, the contract review? Read it now

Missed Part Two, documentation? Read it here

Missed Part Three, the formal review? Read it here