DevOps Experts Share Advice on Measuring Success - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
DevOps
News
8/13/2018
02:00 PM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

DevOps Experts Share Advice on Measuring Success

Watch out for navel gazing when it comes to your DevOps success metrics.

Sears is at risk of bankruptcy, but not for the reasons we’re used to reading about here.

While Sears has been in decline for many years, most recent CEO Eddie Lampert’s plan to sell “member” (employees aren't allowed to say "customer") data from subscription sign-ups floundered when an aggressive customer sign-up campaign called Shop Your Way slowed down check-out lines at Sears-owned stores. This aggravated customers and resulted in a loss of sales and decreased sales per minute, according to reporting by Business Insider. BI also reported that Lampert ignored that the average Sears customer wanted to buy in stores, not online, when he decided to go “asset light” and slow investment in their brick and mortar business.

As is the case with most retailers, it would be easy to point to Amazon as the reason for Sears’ decline, but the problem seems to be more complicated than that. According to reports from current and former employees, Lampert spent too much time dreaming up how he could emulate the digital business model, instead of zeroing in on his customers’ needs.

A business struggling to survive is what can happen when you focus on the wrong metrics and forget that the most important one is customer satisfaction. This failure to see the big picture, or navel gaze, is something that challenges many DevOps teams.

Image: Shutterstock
Image: Shutterstock

In a survey conducted by Interop ITX and InformationWeek in February 2018, 49% of the 150 technology professionals say they measure DevOps success by primarily looking at internal factors such as lower costs, fewer application defects, improved application performance, improved efficiencies, better ROI, and improved collaboration across departments. This number rose from just 35% in 2017.

And it's not a good sign. Andi Mann, chief technology advocate for Splunk finds the data "disturbing." 

"Looking at how modern businesses are disrupting enterprises, it's about delivering services," says Mann. "[As a customer] I don’t care about server response time, I care about how quickly I get a movie."

Only 29% of respondents to the same survey said that when measuring DevOps success, they primarily look at external business factors (increased revenue, faster time-to-market, improved competitive positioning, enhanced customer experience, etc.), which was down one percentage point from 2017.

These business factors help DevOps teams figure out if they’re delivering quality service to the customer, which is the ultimate goal, says Mann.

Robert Reeves, co-founder and CTO of Datical agrees that when looking at DevOps success metrics, focusing efforts on delivering to the customer is key, adding that "you need to evaluate results.”

Reeves says the way you measure results, according to the DevOps Research and Assessment (DORA) report is by looking at these four metrics:

  1. Deployment frequency
  2. Lead time for changes
  3. Mean time to recovery (MTTR)
  4. Change failure rate

So why do a lot of DevOps teams focus on the wrong issues?

One reason, says Mann, is that historically, operations haven't been tied to the business, they’ve been siloed, but DevOps changes that.

“A lot of organizations are still in these functional silos, they measure what they can directly affect, and unfortunately those silos can’t affect outcomes,” says Mann.

In a survey conducted by Micro Focus on implementing DevOps in the enterprise, 50% of respondents said that IT silos are a major issue and are an impediment to their application development.

Once your DevOps team understands the metrics that connect your work to customer satisfaction, look at all the tools your DevOps team uses to deliver for your success metrics, says Rosalind Radcliffe, a distinguished engineer at IBM.

"Many of the tools that people adopt as part of their DevOps transformation help generate data that can help drive numbers into these metrics," says Radcliffe. 

Knowing what to measure and how to measure is half the battle, but what's possibly more important is knowing who needs to raise their hand to keep the DevOps team on track. 

“The glib answer,” says Mann, “is that everyone should care [about DevOps success metrics].” However, the more specific answer Mann gives is leaders of application development or a project management office (PMO) or project manager (PM) for application development at larger companies. If you’re at a smaller company, Mann says a scrum master could be in charge of evaluating a DevOps team’s value stream and success metrics. 

If you’re interested in learning more about the State of DevOps in 2018, check out our report here.

For more on DevOps in the enterprise, check out these recent articles. 

Where is My Job Title in Scrum?

As Iron Sharpens Iron: Hone Your Engineering Team

Python Gains on SAS, R

Why Place Security (Partly) in the Hands of Developers

 

 

Emily Johnson is the digital content editor for InformationWeek. Prior to this role, Emily worked within UBM America's technology group as an associate editor on their content marketing team. Emily started her career at UBM in 2011 and spent four and a half years in content ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Slideshows
What Digital Transformation Is (And Isn't)
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/4/2019
Commentary
Watch Out for New Barriers to Faster Software Development
Lisa Morgan, Freelance Writer,  12/3/2019
Commentary
If DevOps Is So Awesome, Why Is Your Initiative Failing?
Guest Commentary, Guest Commentary,  12/2/2019
White Papers
Register for InformationWeek Newsletters
State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Video
Current Issue
Getting Started With Emerging Technologies
Looking to help your enterprise IT team ease the stress of putting new/emerging technologies such as AI, machine learning and IoT to work for their organizations? There are a few ways to get off on the right foot. In this report we share some expert advice on how to approach some of these seemingly daunting tech challenges.
Slideshows
Flash Poll