The state of DevSecOps - where are we at with application security?

Merging the security function into DevOps won't happen overnight, but it's a vital step and the barriers aren't as high as sometimes believed

DevOps is easing into middle age, a solid, mature and well-understood methodology for anyone creating software - particularly containerised, microservices-based software in the cloud. Nevertheless, as an ongoing process with no fixed destination, DevOps means different things to different people. But there's one thing nearly everyone agrees on - there's a need to integrate security testing in all parts of the software development lifecycle (SDLC).

This requires some serious change, because the way security has traditionally been practised, largely at the end of the lifecycle, does not fit in at all well with the rapid iterations beloved of DevOps teams, with the result that security is sometimes seen by the DevOps team as a bureaucratic bottleneck, an unloved and undervalued police force, a slow-moving blocker standing in the way of software success.

With DevSecOps, instead of being a final checking procedure in which applications are tested for a large number of different vulnerabilities, the idea is to carry out security tests at every step: when the developer's writing code, security is there; when she pushes it to a test branch, security's there too. Security's there when a build is added to the test server, and again when it makes it to the staging server. When the app gets deployed to production, there it is again, and while the application is running security is there too.

But hang on a second. Are security people supposed to be everywhere at once?

Well no, because the idea is that DevOps folks do much of the security analyses themselves, with you showing the way. At each stage they check that the code is passing the tests before taking it to the next one. This is hugely beneficial because the further along the chain a bug gets the more damage it can do and the more expensive it is to correct.

DevSecOps in practice

So that's the theory. As always with DevOps, and indeed with most things tech, it's hard to fault the concept, but putting it into practice may be a whole different ballgame because it involves the C-word: culture. It means changing peoples' roles and possibly their status too, and that always gets political.

To find out how DevSecOps is working out in the real world, we asked 80 UK IT professionals from the Dev, Ops and Sec camps for their opinions. Most of these IT professionals were engaged with DevOps. Sixty per cent of our respondents were already using DevOps with another 20 per cent getting started. Of the DevOps adopters, half were also in the process of adopting DevSecOps with another 26 per cent considering it. Those adopting DevSecOps were working disproportionately in larger organisations and in the technology and public sectors

The main benefits perceived by the respondents were better product security, early identification of flaws and faster deliver.

Rank the following benefits of DevSecOps to an organisation like yours

• DevSecOps should strengthen overall product security
• Most vulnerabilities can be identified in the early stages of development where they are easier and cheaper to tackle
• Faster software delivery
• Security teams more efficient and consistent
• Easier to comply with regulations
• Security load shared between multiple teams

It's easy to see how embedding security in the development process would help meet these aims, but historically organisations have often struggled to bring security into the fold. That's because cultural change, and as adopters of DevOps have already found out, doesn't just happen overnight - particularly in larger organisations where breaking down silos can bring most benefits.

Getting past the stereotypes

Where you have different cultures unfortunately you also have stereotypes. Here are some in this space.

The DevOps engineer

Coders code. That's what they are there for. They move fast with that code, they take risks, they multitask, they innovate, they optimise, they automate, but they rarely raise their heads from their screens to consider the wider world. In particular, they don't have time to learn about all the hundreds of CWEs and CVSs and the latest threat patterns - they have enough to do keeping up with their rapidly changing repos, languages and frameworks.

The software security professional

Security folks are slow and picky. They see themselves as the ultimate bulwark between the company and the chaotic world outside. Secretly they love finding all the bugs and errors that the sloppy coders have introduced, because then they can feel good about exerting their policing power.

These are crude stereotypes, based on movie clichés as much as anything else, but as a result of their different priorities and backgrounds there certainly are cultural differences between DevOps engineers and security professionals. But are these differences really so hard to surmount?

The team seem to appreciate learning new things and being able to guide the development - Senior analyst, manufacturing

No, said our respondents, when we gave them a mix of positive and negative scenarios about the wants and needs of security and DevOps professionals. Security folks are itching to lay down the law, right? Wrong. Few people thought security enjoy being 'the police department' (although a small number did even among the security professionals - showing there may be a grain of truth behind the stereotype). However, larger numbers believed that security professionals do fear that getting involved in DevSecOps might simply mean extra work for them.

But far more respondents felt that security people see closer involvement with developers and operations as a positive. They're tired of being the ‘Department of No', they're keen to learn new and relevant skills, and they're happy to be part of a cross-functional team.

The tunnel-visioned dev stereotype is also wide of the mark. Few people believe that devs see security as outside of their remit. Again, though, there are some fears that increased involvement in security will inevitably mean more work and that it might slow them down - a cardinal sin in development circles. Once again, however, far more respondents picked up on the positives. Building in security at every step will reduce the frustration of builds being rejected. Plus, developers often love learning new skills, with many seeing it as an opportunity to further their careers.

"The team seem to appreciate learning new things and being able to guide the development. Adding security has given them a sense of control and understanding" commented a senior analyst in manufacturing.

So, the two sides are apparently not so far apart after all. A little concerted effort, care and attention to ensure no-one gets overloaded - both devs and security folks are notoriously busy people after all - and the hurdles should be small ones. Much of it is about good communication and avoidance of finger pointing.

So what's the most effective way to get security people involved in DevOps? According to our respondents it's to have security professionals create policies and procedures that are easy for devs to understand. Second was to automate security processes and tools, with security taking the lead; and third, make security a full part of the DevOps team. Unluckily for security folks, giving them a payrise was the least popular option.

What do you think are the most effective ways to get security people involved in DevOps?

• Create security policies and procedures that are easy for Devs to understand
• Automate DevOps security processes and tools with security taking the lead
• Make security staff part of the same team
• Move coding and security onto a common platform
• Have security people train DevOps engineers on an ongoing basis
• Demonstrate vulnerabilities through discovery/scanning
• Teach security people to code
• Give security staff a pay rise

The highest-ranked suggestions are in the hands of their security professionals. Security can cease to be the 'Department of No', becoming a facilitator instead. It's up to them to explain their craft and educate the developers (obviously the devs must be receptive). This includes choosing appropriate security tools and explaining what the output from those tools means.

The tools

As usual with DevOps there is a wide choice, from the cost-free to the expensive, from platforms to utilities, with different tools covering the different stages of the SDLC and a few platforms that claim to cover all of them.

At the start of the SDLC we have static application security testing (SAST) tools which check the code as it's being written, the 'static' indicating that the testing is done before the application is up and running.

The platform based GitLab (46% said they use it) and GitHub Code Scanning (41%) were the most popular options here among our DevSecOps cohort, then there was a dead heat between Veracode and WhiteSource (13%).

GitLab (50%) was also popular at the next stage too, along with Qualys (21%) and OWASP ZAP (16%). Dynamic application security testing ( DAST) tools check the built application for vulnerabilities rather than the code itself. They are used to detect vulnerabilities affecting a web application that has been deployed, sending alerts to the security team when flaws are found.

And it was GitLab (44%) again when it came to checking third-party open source libraries for vulnerabilities. Software composition analysis software identifies the open source software in a codebase and checks for security, licence compliance and code quality, very important when applications may be constructed from tens or even hundreds of different sources. Microsoft's SCA tool for Azure cloud was second most popular (28%).

Finally, we looked at runtime application self-protection tools, which were very much a minority play (71% said they don't use them). Embedded in the application, RASP tools analyse software from the inside while it is running, enabling applications to automatically block unusual inputs, suspicious activity and possible cyber attacks as they are occurring, and protecting the runtime. Imperva (19%) was the top choice here.

Except for RASP tools, which live inside the app, a platform approach was the most popular among our respondents. It's always good to have a choice, of course, but tools can become islands and for many, it seems, consolidation makes sense, just as it does to consolidate the application security function within DevOps. It may take a while, as did merging development and operations, but our study shows that most teams see the logic in this and are making moves towards DevSecOps, and also that the barriers may not be as high as some might think.