Measuring DevOps: The Key Metrics That Matter - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
DevOps
Commentary
3/2/2017
02:00 PM
John Edwards
John Edwards
Commentary
Connect Directly
Twitter
RSS
50%
50%

Measuring DevOps: The Key Metrics That Matter

Determining DevOps' ultimate value to an organization is essential for receiving maximum benefits.

Metrics bring a new level of clarity and insight to enterprise DevOps programs, says Anders Wallgren, chief technology officer of Electric Cloud, a DevOps optimization software company based in San Jose, Calif.

DevOps is being adopted by organizations worldwide for several well-understood and proven reasons. Most importantly, the approach accelerates the software development and release cycle, improves the quality of deliverables and leads to problems being solved more rapidly. That much is obvious. Still, for many adopters, DevOps' benefits, while readily apparent and effective in day-to-day operations, remain frustratingly intangible. To obtain an accurate assessment of DevOp's actual RoI, and to know how to get the most value out of a new or existing DevOps program, an organization needs to find a way of calculating its impact on the bottom line.

Wallgren, will who lead the conference session "Measuring DevOps: The Key Metrics that Matter" at Interop ITX on May 19, believes that just a few basic metrics can help managers better understand just how much value the approach is bringing to their organization.

Image: EME/Pixabay
Image: EME/Pixabay

According to Wallgren, DevOps adopters are beginning to take a more rigorous approach to measurements that quantify benefits. "It's to learn how we communicate, how we pass information from one organization to the other and how we make sure that we’re delivering value to our customers," he says.

The first step in creating effective DevOps metrics is identifying and selecting measurement categories. Two prime categories are efficiency and effectiveness. "Efficiency shows how much it costs to do something, such as deliver a software release or deliver a piece of functionality, and how long it takes to do it," Wallgren says. Effectiveness, on the other hand, presents results. "Did what we do help? Did it work?  Did the customer use it, enjoy it, get value out of it?" Wallgren says.

Efficiency and effectiveness measurements can be applied to people (calculating such things as turnover, capability, and response time), processes (measuring activities like development-to-deployment lead time, deployment frequency, and response time) and technologies (measuring factors like uptime and failure rates).

Wallgren notes that specific metrics can be broken down to provide insights into many different DevOps areas. "You really have to delve into it, look at where the numbers are coming from and what the implications are," he says. "Then chose the ones that are actionable to you."

Andres Wallgren, CTO of Electric Cloud
Andres Wallgren, CTO of Electric Cloud

Regardless of what's being measured, key performance indicators (KPIs) will always evolve over time, Wallgren says. "Once you start getting better at certain things, you’re going to want to look for new KPIs, and the KPIs that you picked early on can be fixed so they stay stable over time and maybe even improve."

Knowledge and preparation are key to a successful DevOps metrics initiative. "For organizations that already understand the processes, doing metrics can be a straightforward and easy thing," Wallgren says. "For organizations that don’t necessarily know or understand the processes, or already have ad hoc processes in place, it can be difficult to start accurately measuring things."

John Edwards is a veteran business technology journalist. His work has appeared in The New York Times, The Washington Post, and numerous business and technology publications, including Computerworld, CFO Magazine, IBM Data Management Magazine, RFID Journal, and Electronic ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/9/2017 | 5:58:20 PM
How readily does your code measure up?
The traditional measure of programmer productivity was how many lines of code did they produce, modified slightly to become how many error-free lines of code did they product. That of course had little to do with how efficiently the code ran, how frequently it broke down or what resources it took to fix it. We far beyond the "how many lines" metric but wise and balanced metrics remain hard to define and implement. Ease of update, ease of integration with production systems would be good ones.
News
IBM Puts Red Hat OpenShift to Work on Sports Data at US Open
Joao-Pierre S. Ruth, Senior Writer,  8/30/2019
Slideshows
IT Careers: 10 Places to Look for Great Developers
Cynthia Harvey, Freelance Journalist, InformationWeek,  9/4/2019
Commentary
Cloud 2.0: A New Era for Public Cloud
Crystal Bedell, Technology Writer,  9/1/2019
White Papers
Register for InformationWeek Newsletters
State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Video
Current Issue
Data Science and AI in the Fast Lane
This IT Trend Report will help you gain insight into how quickly and dramatically data science is influencing how enterprises are managed and where they will derive business success. Read the report today!
Slideshows
Flash Poll