Measuring the Performance of Enterprise Architecture - InformationWeek
IoT
IoT
DevOps
Commentary
1/11/2017
10:30 AM
Imre Kabai
Imre Kabai
Commentary
50%
50%

Measuring the Performance of Enterprise Architecture

Enterprise architecture teams often struggle with measuring and communicating the value of their function. We have implemented a set of simple and straight-forward performance indicators to address this problem.

Do you remember the TV ad from BASF? “We don’t make a lot of the products you buy, we make a lot of the products you buy better”. The predicament of enterprise architecture is similar: We do not make products -- business value generating information technology solutions -- but rather make them better.

In order to create good performance indicators (KPIs) the purpose of the function has to be defined. Herein lies the first challenge.

The architecture function differs from company to company based on culture, funding levels, the role information technology plays in the enterprise, and several other factors. Here is a definition that should would work for many organizations:

As-Is State: The EA function makes the current state of business capabilities, related information, and underpinning technologies visible. This is important so companies do not have to re-discover what they have every time something needs to be changed, interfaced with, or fixed. Understanding as-is states to a sufficient degree is the foundation for alignment, agility, and simplification.

Governance: The function sets rules about the ways the architecture is allowed to change. This is often accomplished by creating principles, standards, and guidelines.

To-Be State: The function creates plans to guide the long-term evolution of the architecture. Business strategy, constraints, current state, and technological opportunities are combined into three-year roadmaps. This is where business strategy meets information technology based innovation.

Assurance: The function makes sure that architecture related decisions follow the rules and roadmaps laid out in Governance and To-B State. This can be accomplished using design reviews.

We can call the four interrelated components the Enterprise Architecture Lifecycle and set out to establish our KPIs:

Good KPIs should be SMART and easy to understand for everybody involved.

As-Is State: The goal is to maintain sufficient coverage of business and technology assets, their important attributes, and relationships.

We have built a lightweight collaboration tool for this purpose. It is populated by periodic application and infrastructure scans. The differences between the scans and the data in the tool trigger workflows and subject matter experts establish the relationships and maintain the attributes.

Our KPI is the percentage of documented assets versus scanned assets and our goal is to keep this number above 90%. 

Governance: Our goal is to have a clear set of agreed upon rules governing the changes of our technology assets. At the beginning of each quarter we assemble a list of principles, standards, and guidelines to be created or updated. Then we prioritize them and select those that we can tackle given the available resources. 

Our KPI measures the percentage that gets created. We strive to reach at least 90% by the end of each quarter.

To-Be State: The goal is to maintain a near complete set of plans for our enterprise technology assets. We plan three years ahead and indicate the desired quarterly state of the assets. We also include the technologies we are planning to introduce in our environments and include detailed rationale for our decisions.

Our KPI measures the percentage of assets with roadmaps and we keep this number above 90%

Assurance: The goal is to make sure that our rules and roadmaps are followed when we design and implement solutions. We use design reviews to evaluate design quality and generate quarterly aggregated reports based on the scores.

Our KPI measures the aggregated design quality and should reach at least 85%, which is B+ level.

 Bonus KPI: We want to see how we measure up to other enterprise architecture shops, so we run annual Gartner benchmarks. We plan on reaching a certain level that makes sense for us.

In Summary

Dear reader, you have probably noticed that the holy grail of the enterprise architecture KPIs is missing. How much business value does the enterprise architecture function generate? This KPI can be decomposed in many ways:

  1. How much value do we generate because we have the necessary business insight through well integrated systems and clean, properly governed data?
  2. What is the value of speed, due to our solutions being built with agility in mind?
  3. What is the value of a clean, well-managed technology portfolio?
  4. What is the value of critical business capabilities relying on high quality technology components?
  5. How much is the worth of solutions that are simple, intuitive, and a pleasure for our users?
  6. What is the value of technology based innovation?

Well, we haven’t figured out the way to measure the business value of enterprise architecture yet, but one thing is for sure, if we wish to get there, it has to be a joint business-IT undertaking. I would love to hear from you if you have good ideas or solutions.

Imre Kabai is director and chief architect at Granite, a $2.5B heavy construction company. Previously he worked as the enterprise architect of Stanford Healthcare, and chief architect of the SLAC National Accelerator Laboratory. His interests include enterprise architecture, ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
100%
0%
Charlie Babcock,
User Rank: Author
1/11/2017 | 6:44:07 PM
Achieving value in enterprise architecture
Imre Kabai has come up with a clear way to think about a complicated subject: enterprise architecture. It's hard to assign value to an individual application process within that architecture, harder yet to assign value to the architecture as a whole. But any experienced operational manager can sense how much value lies in sticking to these guides and rules. They lean toward combating spot fixes, ad hoc cures and freelance work-arounds, all of which contribute to ongoing complexity. 
[Interop ITX 2017] State Of DevOps Report
[Interop ITX 2017] State Of DevOps Report
The DevOps movement brings application development and infrastructure operations together to increase efficiency and deploy applications more quickly. But embracing DevOps means making significant cultural, organizational, and technological changes. This research report will examine how and why IT organizations are adopting DevOps methodologies, the effects on their staff and processes, and the tools they are utilizing for the best results.
Register for InformationWeek Newsletters
White Papers
Current Issue
IT Strategies to Conquer the Cloud
Chances are your organization is adopting cloud computing in one way or another -- or in multiple ways. Understanding the skills you need and how cloud affects IT operations and networking will help you adapt.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll