Measuring the Performance of Enterprise Architecture - InformationWeek
IoT
IoT
DevOps
Commentary
1/11/2017
10:30 AM
Imre Kabai
Imre Kabai
Commentary
50%
50%

Measuring the Performance of Enterprise Architecture

Enterprise architecture teams often struggle with measuring and communicating the value of their function. We have implemented a set of simple and straight-forward performance indicators to address this problem.

Do you remember the TV ad from BASF? “We don’t make a lot of the products you buy, we make a lot of the products you buy better”. The predicament of enterprise architecture is similar: We do not make products -- business value generating information technology solutions -- but rather make them better.

In order to create good performance indicators (KPIs) the purpose of the function has to be defined. Herein lies the first challenge.

The architecture function differs from company to company based on culture, funding levels, the role information technology plays in the enterprise, and several other factors. Here is a definition that should would work for many organizations:

As-Is State: The EA function makes the current state of business capabilities, related information, and underpinning technologies visible. This is important so companies do not have to re-discover what they have every time something needs to be changed, interfaced with, or fixed. Understanding as-is states to a sufficient degree is the foundation for alignment, agility, and simplification.

Governance: The function sets rules about the ways the architecture is allowed to change. This is often accomplished by creating principles, standards, and guidelines.

To-Be State: The function creates plans to guide the long-term evolution of the architecture. Business strategy, constraints, current state, and technological opportunities are combined into three-year roadmaps. This is where business strategy meets information technology based innovation.

Assurance: The function makes sure that architecture related decisions follow the rules and roadmaps laid out in Governance and To-B State. This can be accomplished using design reviews.

We can call the four interrelated components the Enterprise Architecture Lifecycle and set out to establish our KPIs:

Good KPIs should be SMART and easy to understand for everybody involved.

As-Is State: The goal is to maintain sufficient coverage of business and technology assets, their important attributes, and relationships.

We have built a lightweight collaboration tool for this purpose. It is populated by periodic application and infrastructure scans. The differences between the scans and the data in the tool trigger workflows and subject matter experts establish the relationships and maintain the attributes.

Our KPI is the percentage of documented assets versus scanned assets and our goal is to keep this number above 90%. 

Governance: Our goal is to have a clear set of agreed upon rules governing the changes of our technology assets. At the beginning of each quarter we assemble a list of principles, standards, and guidelines to be created or updated. Then we prioritize them and select those that we can tackle given the available resources. 

Our KPI measures the percentage that gets created. We strive to reach at least 90% by the end of each quarter.

To-Be State: The goal is to maintain a near complete set of plans for our enterprise technology assets. We plan three years ahead and indicate the desired quarterly state of the assets. We also include the technologies we are planning to introduce in our environments and include detailed rationale for our decisions.

Our KPI measures the percentage of assets with roadmaps and we keep this number above 90%

Assurance: The goal is to make sure that our rules and roadmaps are followed when we design and implement solutions. We use design reviews to evaluate design quality and generate quarterly aggregated reports based on the scores.

Our KPI measures the aggregated design quality and should reach at least 85%, which is B+ level.

 Bonus KPI: We want to see how we measure up to other enterprise architecture shops, so we run annual Gartner benchmarks. We plan on reaching a certain level that makes sense for us.

In Summary

Dear reader, you have probably noticed that the holy grail of the enterprise architecture KPIs is missing. How much business value does the enterprise architecture function generate? This KPI can be decomposed in many ways:

  1. How much value do we generate because we have the necessary business insight through well integrated systems and clean, properly governed data?
  2. What is the value of speed, due to our solutions being built with agility in mind?
  3. What is the value of a clean, well-managed technology portfolio?
  4. What is the value of critical business capabilities relying on high quality technology components?
  5. How much is the worth of solutions that are simple, intuitive, and a pleasure for our users?
  6. What is the value of technology based innovation?

Well, we haven’t figured out the way to measure the business value of enterprise architecture yet, but one thing is for sure, if we wish to get there, it has to be a joint business-IT undertaking. I would love to hear from you if you have good ideas or solutions.

Imre Kabai is director and chief architect at Granite, a $2.5B heavy construction company. Previously he worked as the enterprise architect of Stanford Healthcare, and chief architect of the SLAC National Accelerator Laboratory. His interests include enterprise architecture, ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
100%
0%
Charlie Babcock,
User Rank: Author
1/11/2017 | 6:44:07 PM
Achieving value in enterprise architecture
Imre Kabai has come up with a clear way to think about a complicated subject: enterprise architecture. It's hard to assign value to an individual application process within that architecture, harder yet to assign value to the architecture as a whole. But any experienced operational manager can sense how much value lies in sticking to these guides and rules. They lean toward combating spot fixes, ad hoc cures and freelance work-arounds, all of which contribute to ongoing complexity. 
2018 State of the Cloud
2018 State of the Cloud
Cloud adoption is growing, but how are organizations taking advantage of it? Interop ITX and InformationWeek surveyed technology decision-makers to find out, read this report to discover what they had to say!
News
Enterprises to Bump Up IT Spending in 2019
James M. Connolly, Executive Managing Editor, InformationWeekEditor in Chief,  9/12/2018
News
AIOps to Drive Big IT Pivot
Jessica Davis, Senior Editor, Enterprise Apps,  9/11/2018
Commentary
AWS vs. Azure: Users Share Their Experiences
Guest Commentary, Guest Commentary,  9/7/2018
Register for InformationWeek Newsletters
Video
Current Issue
The Next Generation of IT Support
The workforce is changing as businesses become global and technology erodes geographical and physical barriers.IT organizations are critical to enabling this transition and can utilize next-generation tools and strategies to provide world-class support regardless of location, platform or device
White Papers
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Sponsored Video
Flash Poll