Measuring the Performance of Enterprise Architecture
Enterprise architecture teams often struggle with measuring and communicating the value of their function. We have implemented a set of simple and straight-forward performance indicators to address this problem.
Do you remember the TV ad from BASF? “We don’t make a lot of the products you buy, we make a lot of the products you buy better”. The predicament of enterprise architecture is similar: We do not make products -- business value generating information technology solutions -- but rather make them better.
In order to create good performance indicators (KPIs) the purpose of the function has to be defined. Herein lies the first challenge.
The architecture function differs from company to company based on culture, funding levels, the role information technology plays in the enterprise, and several other factors. Here is a definition that should would work for many organizations:
As-Is State: The EA function makes the current state of business capabilities, related information, and underpinning technologies visible. This is important so companies do not have to re-discover what they have every time something needs to be changed, interfaced with, or fixed. Understanding as-is states to a sufficient degree is the foundation for alignment, agility, and simplification.
Governance: The function sets rules about the ways the architecture is allowed to change. This is often accomplished by creating principles, standards, and guidelines.
To-Be State: The function creates plans to guide the long-term evolution of the architecture. Business strategy, constraints, current state, and technological opportunities are combined into three-year roadmaps. This is where business strategy meets information technology based innovation.
Assurance: The function makes sure that architecture related decisions follow the rules and roadmaps laid out in Governance and To-B State. This can be accomplished using design reviews.
We can call the four interrelated components the Enterprise Architecture Lifecycle and set out to establish our KPIs:
Good KPIs should be SMART and easy to understand for everybody involved.
As-Is State: The goal is to maintain sufficient coverage of business and technology assets, their important attributes, and relationships.
We have built a lightweight collaboration tool for this purpose. It is populated by periodic application and infrastructure scans. The differences between the scans and the data in the tool trigger workflows and subject matter experts establish the relationships and maintain the attributes.
Our KPI is the percentage of documented assets versus scanned assets and our goal is to keep this number above 90%.
Governance: Our goal is to have a clear set of agreed upon rules governing the changes of our technology assets. At the beginning of each quarter we assemble a list of principles, standards, and guidelines to be created or updated. Then we prioritize them and select those that we can tackle given the available resources.
Our KPI measures the percentage that gets created. We strive to reach at least 90% by the end of each quarter.
To-Be State: The goal is to maintain a near complete set of plans for our enterprise technology assets. We plan three years ahead and indicate the desired quarterly state of the assets. We also include the technologies we are planning to introduce in our environments and include detailed rationale for our decisions.
Our KPI measures the percentage of assets with roadmaps and we keep this number above 90%
Assurance: The goal is to make sure that our rules and roadmaps are followed when we design and implement solutions. We use design reviews to evaluate design quality and generate quarterly aggregated reports based on the scores.
Our KPI measures the aggregated design quality and should reach at least 85%, which is B+ level.
Bonus KPI: We want to see how we measure up to other enterprise architecture shops, so we run annual Gartner benchmarks. We plan on reaching a certain level that makes sense for us.
Dear reader, you have probably noticed that the holy grail of the enterprise architecture KPIs is missing. How much business value does the enterprise architecture function generate? This KPI can be decomposed in many ways:
How much value do we generate because we have the necessary business insight through well integrated systems and clean, properly governed data?
What is the value of speed, due to our solutions being built with agility in mind?
What is the value of a clean, well-managed technology portfolio?
What is the value of critical business capabilities relying on high quality technology components?
How much is the worth of solutions that are simple, intuitive, and a pleasure for our users?
What is the value of technology based innovation?
Well, we haven’t figured out the way to measure the business value of enterprise architecture yet, but one thing is for sure, if we wish to get there, it has to be a joint business-IT undertaking. I would love to hear from you if you have good ideas or solutions.
Imre Kabai is director and chief architect at Granite, a $2.5B heavy construction company. Previously he worked as the enterprise architect of Stanford Healthcare, and chief architect of the SLAC National Accelerator Laboratory. His interests include enterprise architecture, ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
2017 State of IT ReportIn today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Infographic: The State of DevOps in 2017Is DevOps helping organizations reduce costs and time-to-market for software releases? What's getting in the way of DevOps adoption? Find out in this InformationWeek and Interop ITX infographic on the state of DevOps in 2017.