InformationWeek
Defining The Business Value Of Technology
October 26, 1998
Measure Success
 Print this story |
But these statistics weren't telling the whole story. When Pichereau conducted a random survey of end users, she found that many city employees weren't bothering to call the help desk at all, which meant lost productivity. What's more, users reported their systems were down 41 minutes per week, more than double the world's best average. Indianapolis is working with SCT to improve. "It's one thing to look at statistics, but to get a complete picture, you always need to go back to employees," Pichereau says.
The potential danger with survey statistics and performance information is that an onslaught of technical details can obscure the larger business picture. "If you get the fever, it can get overwhelming. You wind up with metric clutter," says Rick Roscitt, president and CEO of AT&T Solutions. "The client should understand the difference between what's measurable and what's critical."
To get a snapshot of an outsourcing vendor's business impact, some IT managers use a "balanced scorecard," which weighs customer satisfaction, productivity, cost, and quality of service. Tony Hutchings, head of global application delivery quality at J.P. Morgan, uses a scorecard to stay on top of one of the most intricate of IT outsourcing deals. Morgan contracted four companies--Andersen Consulting, AT&T Solutions, Bell Atlantic Network Integration, and Computer Sciences--dubbed the Pinnacle Alliance to manage Morgan's data centers, voice and data networks, and critical human-resources and financial applications. It's a seven-year contract worth several billion dollars.
Morgan's scorecard is an automated reporting system designed in conjunction with members of the alliance and jointly administered by Morgan and its suppliers. It examines actual vs. planned costs, service levels, and staff responsiveness in each outsourced area and for each supplier in the alliance deal. The scorecard also looks at how each partner brings future value to the partnership. In the applications area, for instance, suppliers are expected to bring Morgan's software-development program in compliance with Carnegie Mellon University's Capability Maturity Model. CMM is a five-tiered system that examines the sophistication of a software-development operation and helps develop training, best practices, and checkpoints.
So far, payoffs from the consortium have been impressive. Morgan estimates it has saved $50 million in the first 18 months of its contract with the alliance. Mainframe costs, for example, have plunged as the alliance absorbs total cost of ownership charges.
Look Closer
But neither the alliance nor the means of tracking its performance has been perfect. An audit late last year revealed discrepancies in the scorecard. Customer-satisfaction and productivity statistics in the applications sector were skewed because quarterly measurements of rapid application development projects didn't come soon enough. As a result, Morgan was discovering crises and processes that needed adjustments too late. "The alliance was giving itself more credit than it deserved," Hutchings says. "It wasn't malicious at all, but it was clear that the process needed to be fixed." Hutchings installed a continuous performance-measurement program so Morgan can make changes and set new standards.
For some businesses, complex contracts and metrics just get in the way of a good customer-supplier relationship. Hal Upbin, president and CEO of St. Louis clothing manufacturer Kellwood Co., has an eight-year, multifaceted deal with EDS. "We've been able to reach a level of trust that is just superb," Upbin says. "We have a big [contract] that EDS spent a year analyzing and putting together. We put that signed document on a shelf a year and a half ago and haven't looked at it since."
Though EDS often works with its customers to employ metrics, the vendor prefers to operate from "statements of work" rather than narrowly defined measurements. "There's common expectations," says Marlene Eisenberg, a member of EDS's legal team. "The customer knows what EDS does, and we know what we're required to deliver."
But that approach doesn't always work. A Rocky Mountain high-tech firm thought it had such an understanding with its outsourcing vendor. Three years ago, the company began an outsourcing relationship "in good faith, and we figured we'd decide how to measure later," a company executive says. Unfortunately, neither party got around to doing that.
The result: Expectations were so different that the entire agreement has to be renegotiated. For example, the customer wanted desktop problems resolved in two hours; the vendor thought the next business day would suffice. Says the Rocky Mountain executive, "Gathering data is useful, but it doesn't help if you haven't agreed on what the goal is." Good advice for any deal.
return to page 1
Read sidebar story, "What To Look For In Metrics."
Back to This Week's Issue
Send Us Your Feedback
Top of the Page
