Larry Maccherone Explains How to Rescue a Struggling DevSecOps Program

There are many historic and cultural differences between dev and security teams that persist today – fast vs. slow, risk-hungry vs. risk-averse, focused on profits vs. focused on cost avoidance, and more. Meanwhile, the increasing use of open-source code libraries and cloud-native applications are creating new cybersecurity threats, pressing DevSecOps teams to find new appsec solutions, better observability, and fresh collaboration ideas.

In this archived keynote session, Larry Maccherone, Dev[Sec]Ops Transformation Architect at Contrast Security, explains how to rescue a struggling DevSecOps program.

This segment was part of our live “DevSecOps Essentials that Enable Efficient Security” virtual event. The event was presented by InformationWeek and ITPro Today on October 19, 2023.

View the entire “DevSecOps Essentials that Enable Efficient Security” live webinar on-demand here.

A transcript of the video follows below. Minor edits have been made for clarity.

Larry Maccherone: Even before the term DevSecOps was coined, and I wrote the “DevSecOps Manifesto,” but the concept of the original "build security in" initiative was essentially what we're calling DevSecOps now. But it was pre-DevOps, so, of course, that phrase couldn't do it or use it. But it was more about automation and letting developers sort of have a bigger role in building the security into the product as opposed to waiting and tacking it on or bolting it on after the fact.

Related:Eve Logunova-Parker Details Agile’s Role in the Age of IT Automation

This is the way security has been traditionally done over the years. Now, we're growing from that old, traditional way to a more modern, developer-oriented way. Along that journey, I've gained a lot of experience, meaning I've made a lot of mistakes while doing that. So, the first challenge in fixing, repairing, or improving a DevSecOps program is to get the mindset right.

We often believe that we can achieve DevSecOps, if only we create more relevant and consumable policies. If only we get better at enforcing those policies and implement the right metrics-based incentives. If only we trained better, especially the developers, and worked to collaborate better with engineering via a Champion Program.

Maybe it's all about tools, if only I had the right tools, and integrated them the right way. All these things are on the right track, although I will take a slightly different angle on the policy lines here. That alone won't get you to a program that will go across an enterprise. You need a systematic approach to changing the culture of the organization, both the security and the engineering side of that culture. I'm going to talk to you about what I call a transformation blueprint for DevSecOps. So, this works.

Related:Exploring Cloud Crisis Management Solutions

When we did this at Comcast, we started with an experiment using only 10 engineering teams out of the 600 available. We gather data, in fact, I spent about half of my first-year budget on putting a metric system in place, so we could learn, basically. Do you remember that first slide where I said I've made every mistake in the book?

Well, I had numbers to show when I made a mistake and I also had numbers to show when something we did was really very effective. Even ones that were effective, but not enough to justify the energy we put into them, or not enough to be done before we do something else that was more effective. So, these numbers are important and I'm an analytics guy.

In fact, all those open-source projects are data and analytics-oriented. I spent a lot of energy building the metric and data gathering systems, which is more of the effort to be able to prove that the changes we were making were not just spreading far and wide, that's one metric, but also reducing vulnerabilities of the company reducing risk in production.

The first 157 teams we put through the program had one-sixth as many vulnerabilities as the teams that were not in the program. And you can say, well, maybe there's a selection bias there. We also did within-subjects analysis. So, we had data from the teams before they were in the program, and after they graduated from the program, and the data held true.

It was still about one-sixth of as many, equaling a fivefold to sixfold improvement in risk reduction. The other interesting thing is that the number of resources we needed in security in terms of managing this program was one-fourth as many as the people we needed to manage this program for 600 development teams.

Before, we had about 40 full-time equivalents just doing application security vulnerability management, but by the end of the five-year run, we had zero people doing application security vulnerability management. So, we save that, and we only replace them with about 10 people running the program that I'm going to describe to you here.

So again, we're talking about mindset here. One of the other things that people think is, if we just put some DevOps lipstick on this traditional way of thinking about application security, that's what we'll call DevSecOps. Well, that's not DevSecOps to me. As a matter of fact, I have drifted away from that way of thinking.

I wrote the “DevSecOps Manifesto” all those years ago, but I drifted away from that term, because it's gotten overloaded by many people who use this definition on the left. Just throw some tools at it, do it the old way, but expect developers to do it or just collaborate more. No, that's not what we mean.

What we mean, I think, is better represented by the term developer-centric application security. Sometimes known as "shift left" and now at my current company, Contrast, we say "shift smart."

Watch the archived “DevSecOps Essentials that Enable Efficient Security” live virtual event on-demand today.