DevOps in 2017: too much Dev and not enough Ops

Computing research presented at the DevOps Summit 2017 this morning

DevOps is about bringing the development and operations roles together with the aim of speeding up the production of software. In a broad practical sense, this means changing the culture in the IT department and beyond, but it's also about methodologies, and it's about tools too. During Computing's latest programme of research in this area we looked at the current state of DevOps in all three areas.

This article considers the cultural side, and whether DevOps teams are getting the balance right. Future articles will consider the methodologies and toolsets. For a more detailed write-up of the findings download the Computing DevOps Review 2017.

Who'd be a tester?

In an organisational sense aims of DevOps are rather egalitarian: tear down the wall, no more ‘us and them', everyone in the same boat - or at least the same scrum. We've all got a job to do so let's all muck in and get it done as quickly and efficiently as we can.

That's the idea, but it's been fairly clear from the start that some DevOps-ers are more equal than others. It's called DevOps and not OpsDev for a reason. In many DevOps teams those with most weight, both in terms of numbers and clout are the Devs. Meanwhile Ops see their core skills whittled away, those willing to change becoming honorary Devs with rest being advised to ‘seek opportunities elsewhere', while the poor old test and QA team don't really get to play at all.

"If you don't get the culture right, it's very difficult to get the balance in the teams right," a head of application services in media told us during the research process, and that's surely a statement of the truth. While the management of infrastructure is changing, Ops is still an essential part of the equation.

"For some the implementation of DevOps had been too Dev focused," said a global programme director in utilities. "The realisation can set in that you need to bring in more operational elements."

A head of delivery in the transport sector concurred: "Companies might find they've gone down a very Dev-led road that isn't commercially viable and there is quite a big danger of that," he said.

The balance may not be right, and it's exacerbated by the fact that it's the developers who are most in demand, commanding the biggest salaries, and attracting the most offers from elsewhere.

"When we get a good developer we get our claws into them, we'll do anything to keep them," said a CIO in finance.

Contrast this to the attitude to testers and QA. Variously described during our focus groups and interviews as "speaking a different language" and being of "variable quality" the second-class status of the testing role is clear.

"It used to be Dev and Ops, but now we're talking about DevOps and testers," opined one interviewee, confirming their outsider status.

As well as those responsible being sidelined, underappreciated and perhaps misunderstood, the QA and testing stage of the software development lifecycle (SDLC) was blamed by twice the number of survey respondents as a major source of holdups than the next slowest part - the planning stage. The interesting thing here is the answers given by those actively practicing DevOps looked very similar to those who were not.

The testing bottleneck is one of the issues that DevOps is meant to rectify, by rationalising, standardising and automating as much of the process as possible. However, automating testing and QA may be more difficult in practice than had originally been thought with some anomalies more easily picked up by people.

"There will be some tests that I'm sure you'll continue to do manually, for example log testing," said one interviewee.

Sometimes the delays are due to unavoidable factors, such as the need to comply with regulations.

"If it's to do with finance or legal then the QA and testing goes on forever. If it's something that's actually useful, test QA tends to be done as you use it…" said a network security architect, in the aerospace sector.

Others mentioned that the tools available to QA and testing lag behind those in other areas.

Automation can and should be applied to the more mundane aspects of testing, but automated testing tools and human software testers shouldn't be mutually exclusive concepts. Automated tools will do what they are instructed to do. Human beings have different qualities which, as yet, not replicable by automation. Arming skilled testers with automated testing tools can extend their reach and enable a deeper level of testing involving human perception and experience, as well as speed, replication and re-usability.

Clearly the integration of testers and QA into the team is something that requires more thought.

"You need automated testing, that's critical, but you also need people testing. You need both…" said a CIO in finance.

Rather than seeking Devs, Ops or testers, many organisations are now seeking 'DevOps engineers', allrounders capable of taking on development operational and testing tasks. However, such individuals are thin on the ground, with interviewees complaining of candidates being over-specialised or only interested in the more glamorous, customer-facing parts of the job.

DevOps in 2017: too much Dev and not enough Ops

Computing research presented at the DevOps Summit 2017 this morning

Firms also struggle with embedding the security role into DevOps family.

"How do you integrate DevOps with SecOps, that's what everyone is trying to figure out. We haven't integrated SecOps as part of the development process in our company, and most of those tool sets tend to be very costly…" said a lead developer in insurance.

Difficult as it may be, most recognise the logic in incorporating security into all stages of the SDLC. Treating security as an afterthought, bolting it on to the production end of the development cycle or taking short cuts is considered an unwise approach by just about everyone. However, it happens rather more than the number of people denouncing it would suggest.

DevOps is still a relatively new concept, and some practitioners consider that so long as security is not actually made worse by its introduction, then that is sufficient for now. However, there was a recognition that this will need to change as regulations tighten. The adoption of microservices architectures - breaking applications down to separate functions - and increased automation will both facilitate and necessitate the full integration of security into DevOps - so-called DevSecOps - some believe.

"SecOps is just coding with a security mind-set," said the head of platform delivery in transport. "As we move to a world of more and more microservices and containers with more logins, to be able to analyse that you need more of a DevSecOps-type role, which is a security focus using code to analyse rather than just people."

For now, though, DevSecOps has not really taken off to any extent and the security function remains outside of the core group in most cases.

Is the company DevOps ready?

Key to the idea of DevOps is the production of the minimum viable product (MVP), the piece of software that meets just enough of the requirements to be released with the understanding that improvements and fixes will quickly follow.

Where software is developed for internal use, it is important that the company as a whole is on board with this concept. It's no good just adopting DevOps in IT; the whole organisation must become more agile to match, or a two-speed system will result in which the MVP will sit like the proverbial square peg in a round hole. Instead of being delighted that their software has been delivered so fast, the end user will see a rudimentary interface and will complain about the lack of bells and whistles: "Why didn't you finish it before giving it to us".

How do DevOps practitioners feel their efforts are regarded by their business colleagues? Do they understand the break from the past that it represents? ‘Not bad, but could be better' seems to be the consensus, according to our quantitative study.

Only 21 per cent of our DevOps practitioners believed that they were either fully meeting or surpassing expectations. The greatest proportion of respondents said that they were "somewhat" meeting expectations - a response which is probably best described as lukewarm. Thirty-five per cent stated that they were falling short of expectations to varying degrees.

"We're going the DevOps route only to find that legal and finance departments are not agile. Their processes just don't fit," explained one CIO.

The whole organisation must become more like DevOps, said another.

"Some of the tenets of DevOps have got to permeate out through the company, but without that label on it."

So, the principle of DevOps has not yet been fully accepted throughout the business and customer base. We should not be too surprised by this. Old habits are hard to break, DevOps is still new and rollout incomplete in many cases, and techies are not always the best communicators. Part of the cultural shift required for success is explaining better what the team is doing and why.

Download the full Computing DevOps Review 2017