How IT Success Metrics Are Changing

Business and IT have become so intertwined, their goals must align if companies want to stay competitive. That means traditional IT KPIs simply aren’t enough anymore.

"CFOs, COOs and CIOs are the only three organizations that are held accountable [for] forecast ability, budget reductions and resource reductions. None of those three things allow for innovation," said Mike Guggermos, CIO of smart email platform provider Black Pearl. "The only way you get around that is you have to have a leader who recognizes that strategic layer and embeds IT and the CIO into those strategic deliverables for the company, be it from a sales perspective or a productivity perspective or whatever."

Over the past couple of decades, CIOs and others in IT have been told they need to understand the business. To demonstrate that understanding, they need to translate what IT does into business terms. In addition, their goals and KPIs should be tied to business objectives.

"A lot of people believe it's just a matter of cherry-picking operational metrics and bringing those to executive decision-makers," said James Anderson, senior director analyst at Gartner. "They're all reporting on the wrong things."

James Anderson, Gartner

The right thing to report is IT's impact on the business. For example, instead of reporting system downtime alone, one automotive manufacturer's IT department now reports system downtime in terms of inventory that wasn't produced.

"The most effective way to measure [IT] success is by aligning with the business objectives," said Bill Kirwin, managing director, International Institute of IT Economics and an analyst with The Analyst Syndicate. "[T]ake your business objective and look at it as a holistic set of requirements, including the underlying IT functionality, the business process capabilities and all the things within the business that go wrong that are not necessarily IT oriented. [Identify] the systems we're going to put into place in order to meet this business objective, and each of those IT systems will, have key performance indicators that indicate whether they [helping to] meet those business objectives."

Bill Kirwin, IIIE

Tying IT metrics to business outcomes requires a mindset shift

Shifting from traditional IT KPIs to business outcome-oriented KPIs requires cultural change throughout the organization. Business roles need to consider IT a business partner instead of a service provider. Conversely, IT needs to adopt a business mindset and adapt its success metrics accordingly.

Black Pearl's Guggermos said one sure sign IT metrics aren't right is when other C-suite executives ask to see an IT roadmap.

"The more closely intertwined the KPIs are for the senior leadership as a whole, the better off your organization is," said Guggermos. "Don't ever talk to me about the IT roadmap. What are we trying to do? What are the [business] goals over the next six, 12, 18, 36, 48 months, and then we'll put things in place to go support that. It's a business enablement roadmap."

Mike Guggermos, Black Pearl

Before Timothy Wenhold agreed to join Power Home Remodeling as CIO in 2012, he negotiated a couple of changes that sparked a cultural shift and contributed to impressive growth.

"The IT department was viewed more as a utility and support function than a business advantage. I was determined to change that perception, [so] I asked for two things: first, that my title would be Chief Innovation Officer, not the traditional CIO title, and second, we'd rename IT the 'business technology' (BT) department," said Wenhold.

The BT department goes beyond processing help tickets and change requests. It's an entirely new function that's solely focused on delivering products that fuel business growth. The department's success is measured by outcomes rather than outputs."

To align the BT department more closely with the business, Wenhold created cross-functional teams and embedded "producteers" (product managers) in each department, including sales, customer development, HR, accounting, finance, and the contact center. In addition, new hires must complete a two-week onboarding experience in which they observe how their colleagues do their jobs and interact with the technology. Because of those changes, Wenhold said the products BT developers create are more useful to the people using them.

Tim Wenhold, Power HRG

"We grew from $172 million in 2012 to $720 million in 2018 because we've chosen to be a technology company that provides home remodeling services," said Wenhold. "One of the biggest barriers we encountered early on was that we were using an entirely different lexicon than our partners in the business. For that reason, we were working toward an entirely different set of goals. What was important to the business had to become important to us [because] it's the entire reason we exist. When we did, BT became more effective."  

Gartner's criteria for a good metric that drives business decision making

IT dashboards tend to be volume-based. They reflect IT operations instead of the downstream impact. Gartner's Anderson said his firm identified seven characteristics of a metric that impacts business decision making, which apply to IT. They are:

#1: The metric has a clearly defined and defensible causal relationship to a business outcome.

If the goal is to improve a product value chain, the supporting systems must be able to produce the product and turn it into revenue.

"Boards of directors are concerned primarily with revenue, cost, and the risk of maintaining the revenue," said Anderson. "IT should be thinking about the things that are relevant to that, not just how busy IT is (how many intrusions, how many patches, how much uptime, etc.)."

#2: The metric is a leading indicator.

A predictive metric is more interesting than a historical one.

#3: The metric is tailored to the organization.

No company's metrics are a magic formula for success. Copycat tactics don't work because each organization is unique.

"A lot of clients ask me to look at their dashboard or want me to tell them what their metrics and KPIs should be," said Anderson. "Everybody thinks that somebody else has that magic dashboard."

#4: The metric addresses the intended audience's business decision making.

Consider the business decisions people are trying to make. How does what IT does help them address their responsibilities?

#5: The metric reflects the language of the audience.

Being a multilingual CIO or IT professional is important. If you want others to value what you do, speak their language.

"If we're only speaking the language of IT, people that speak finance and business outcomes stop listening to us," said Anderson. "A lot of times the things that we're presenting are the abbreviations or acronyms for things that people don't really understand."

#6: Dashboards should drive action.

Dashboards displaying the status of metrics should not just provide information, they should provide actionable insights.

#7: The metric has a business context.

This goes back to the downstream causal impact mentioned in #1.

"We get there by almost reverse engineering," said Anderson. "Find those metrics, KPIs or indicators that will help us make decisions to course correct or continue doing what we're doing."

Bottom line

Business success hinges on the effectiveness of IT. However, the definition of "effectiveness" has shifted away from just bits and bytes to the strategic goals the organization seeks to achieve. To get there, CIOs and their teams need to translate what they do in to business benefits and measure their success by business outcomes instead of how much work they do.