InformationWeek.com

Welcome Guest. | Log In| Register | Membership Benefits

October 26, 1998

Measure Success

Careful metrics help businesses get what they need from their outsourcing deals

By Noah Shachtman

ow do you get the best deal from your outsourcing vendor? IT executives, industry analysts, and even most vendors agree that tightly defined metrics are the key to prioritizing goals and setting standards when you farm out your network, data center, application development, or help desk.

Across industries, businesses have seen demonstrable performance and efficiency paybacks by working with their outsourcing vendors--and, in some cases, independent consultants--to develop sensible metrics. Companies have taken different paths in deciding which areas to examine and which measurement methods to use. Some, such as Bell Atlantic Corp., look at how well their vendors stack up against stringent performance standards. The city of Indianapolis, meanwhile, complements this data with employee surveys. Some companies, including J.P. Morgan & Co., use an automated reporting system that combines a variety of measurements into a single picture of an outsourcer's business value.

Ironically, successful metrics begin with a decision to stay out of the vendor's way. "Often, people confuse outsourcing, where you turn a process over to a supplier, with contracting, where you control the process and give a supplier tasks," says Peter Bendor-Samuel, president and CEO of IT consulting firm the Everest Group. "You select an outsourcer because you believe their process works. Rather than tell them what to do, use metrics to define the result you want and then measure to see if that result is achieved."

The catch is that these results can't be worded too broadly. Instead of "make sure my mainframe works," set a series of tightly defined goals such as "make sure my mainframe is up 99.9% of the time during business hours." This helps establish performance expectations and priorities.

That's what's being done at Bell Atlantic. The company has a four-year deal, valued at more than $1 million a year, for Keane Inc. to maintain and enhance 15 of Bell Atlantic's legacy purchasing and inventory systems. Under the contract, Keane is ultimately responsible for 4 million lines of Cobol code residing on an IBM 3090 mainframe.

Brian McMillan, director of sourcing and logistics systems at Bell Atlantic, examines Keane's service levels and work distribution to ensure that the partnership is running smoothly. Service-level metrics include making sure mainframe applications are available more than 99.8% of the time during business hours, and that nightly batch-processing jobs are successfully completed at a 99% rate. Standard work-distribution metrics expect about 120 hours of user support and 400 hours of maintenance per month.

By assessing these availability and batch-job rates on a daily basis, McMillan was able to catch several potential problems before they grew. Earlier this year, for example, purchasing and inventory jobs were running long because of a sudden spike in demand, slowing systems throughout the company. Thanks to careful monitoring, Keane redirected manpower into performance improvement, and rewrote purchasing and inventory programs to account for unexpected demand increases.

Catching such discrepancies would have been impossible without a thorough understanding of how these applications are supposed to perform--knowledge far too few of those managing outsourcing partnerships have, experts say. "Too many organizations don't have the data or the means to compare costs in-house with the costs of using outsourcers," says Bruce Guptill, a VP and research director at Gartner Measurement.

McMillan was also able to revise Keane's work schedules to Bell Atlantic's benefit. Last year, he noticed that fewer maintenance requests were coming in, and nearly a third of Keane's contracted work hours were going unused. So McMillan met with Keane managers, and they decided to use the extra hours to convert Cobol programs in preparation for year 2000 compliance.

Power Of Metrics
Metrics have also helped the city of Indianapolis/Marion County government assess the performance of its IT outsourcer, Systems & Computer Technology Corp. SCT runs the city's help desk and manages its 3,500 PCs and 65 Compaq servers. Chris Pichereau, deputy director and contract compliance officer in the city CIO's office, worked with independent consultant Compass America Inc. to review standard metrics, including total cost of ownership and help-desk response time, every two weeks.

These measurements were compared with an average of the world's best performers in these areas. The numbers showed Indianapolis doing well: The average annual cost per PC was $3,410, close to the world's best average of $3,380. Problems phoned into the help desk--about 5,700 per month--were resolved on the first call more than 99% of the time.