Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.
The Forgotten Secret to DevOps Success: Measurement
Measuring the end-to-end DevOps value stream is the key to delivering value and tracking its ROI and allocating resources.
6 Min Read
Tim Buntel
The enterprise DevOps market is shaping up to be one of the fastest growing markets in technology. According to Gartner it reached $2.3 Billion in 2015, up 21% over 2014. As companies invest more in digital transformation to stay relevant, the need to build their own software to drive growth is paramount.
However, a recent study from Forrester Research and Blueprint paints a confusing picture: While DevOps is anecdotally adding value, 50% of practitioners struggle to link DevOps with positive ROI. How can that be the case when so many believe DevOps is no longer a nice-to-have but a need-to-have in order to stay in business?
Measuring the end-to-end DevOps value stream is the key to delivering value and tracking its ROI. Since IT teams can focus on only a few initiatives at a time, it’s critical to identify existing constraints and focus on the most important work to accelerate your transformation and deliver true business value with DevOps. The problem is, many organizations just don’t know where to start. To truly understand and accelerate your DevOps ROI, you need to measure key outcomes, drive improvements through core capabilities, and then identify priority areas and track progress over time.
Image: Shutterstock
Below are a few tips to get you started.
Focus on key outcomes
We’ve established that measurement is the missing link to DevOps ROI, but the crux of the issue is that many teams get hung up on which metrics to evaluate. Based on our research and client work over the past years, we’ve identified what we believe to be the types of metrics that give the best insight into DevOps ROI:
Throughput: Metrics like deploy frequency and lead time for changes, which measure how often you can push code into production or have it be in a deployable state, are good indicators of how efficient your overall DevOps process is.
Stability: Stability metrics such as mean time to recover (MTTR) from a deployment failure and change fail rate (the lower the better) help you gauge the reliability of your software delivery.
Sustainability: Sustainability takes into account factors such as deployment pain and burnout, which go hand in hand. With deployment pain, there is often a lack of confidence in the process. If your teams are always asking, “Is this going to work?” before every deployment, you know there is a problem. That uncertainty takes away from their career fulfillment and enjoyment, ultimately leading to burnout.
We see no tradeoffs among the outcomes listed above. In fact, they all go together. Our experience and research bear this out. The highest performing organizations are able to optimize all these factors.
Drive improvement through core capabilities
Once you’ve established a baseline for the key outcomes above, the next step is to drive improvement. This is done through investment in core capabilities that enable an organization to deliver code with high throughput, stability and sustainability. These capabilities fall into four categories: technology and automation, management and process, measurement and monitoring, and culture. Let’s take a closer look at each.
Technology and automation accounts for various factors like version control, automated deployment and continuous integration. Investments in these capabilities will ensure repeatability of key functions and streamlining of your processes, which will make it easier for everyone involved to be successful.
Management and process includes activities such as gathering and implementing customer feedback so developers know why they are coding what they’re coding, and establishing more streamlined approval processes.
Measurement and monitoring across applications and workflows creates vital information used to inform business decisions. This is where the visualization of work is done via dashboards. By making this information easily digestible, you’ll not only be able to increase the quality of your work, you’ll have a better time communicating it across your organization, which is needed to drive digital transformation.
Culture transformation results in a generative workforce that encourages and supports learning, facilitates better collaboration, and provides resources and tools to make work meaningful.
If you invest in and improve these core capabilities, you will drive improvement in your ability to deliver software. That in turn drives, and positively impacts, your bottom line. (The State of DevOps Reports from 2014 through 2017 support this finding.)
Identify your priorities
The next step in the DevOps measurement process is to identify your priorities. Let’s be realistic for a moment; teams can only focus on a few initiatives at a time. The key is to identify your constraints and work on just a few capabilities at a time. This allows the organization to dedicate resources strategically, and enables teams to focus time and attention on the most important work. By taking a strategic, focused approach, you can accelerate your technology transformation.
[DevOps is on the Interop ITX agenda again. The Interop team has issued a Call for Speakers, including practioners and independent experts who wish to share their experience.]
We do urge you to beware of the maturity model mindset. Improving these capabilities leads to better outcomes, but there is no “level 5,” no ideal state at which you should measure them and declare, “We’re done!” Once you see progress, identify your next constraints and invest again. Always strive for continuous improvement.
Track progress over time
Now that you’ve set the key outcomes to measure, invested in core capabilities, and adopted a continuous improvement mindset, it’s time to set up a process to track how you’re performing over time. This will allow you to track your progress, catch and capitalize on value quickly, and, perhaps most importantly, catch and stop non-value added activity before too much money and time are wasted on it. Tracking progress over time also allows you to find new priorities and constraints that can be fed back into your continuous improvement paradigm. This is the key to realizing the value of your investments in IT.
Don’t get overwhelmed or discouraged if you’re not seeing significant results immediately. The key with large DevOps gains is small improvements over time. They all add up!
Ultimately, measurement is a skill that DevOps pros need to cultivate. By implementing the above strategies, you will be better equipped to capitalize on the most valuable areas for your business, successfully communicate those results across the organization to inspire continued digital transformation and ultimately connect the dots from DevOps initiatives to the bottom line.
Tim Buntel is Vice President of Products for XebiaLabs. For more than 20 years, Tim has been involved with software product management, strategy and marketing at companies including Atlassian, Adobe, Microsoft and Allaire, with an emphasis on developer tools and platforms. He is also a longtime startup mentor and advisor, and public speaker at industry events worldwide.
{image 2}
Dr. Nicole Forsgren is CEO and Chief Scientist for DORA. She is an IT impacts expert who is best known for as the lead investigator on the largest DevOps studies to date. She is a consultant, expert, and researcher in knowledge management, IT adoption and impacts, and DevOps. Her work has been featured in various media outlets and several peer-reviewed journals. She holds a PhD in Management Information Systems and a Masters in Accounting.
About the Author
You May Also Like
More Insights
