InformationWeek

The intangible benefits of technology are emerging as the most important of all

By Bob Violino

hat's the best way to assess the business value of information technology? Talk to 20 financial analysts, economists, and CIOs, and you'll probably get 20 different answers. But a new way of thinking about IT's return on investment is taking hold. Pushed forward by several academics and analysts, and adopted by a cutting-edge group of IT managers, this new philosophy of ROI says it's the things that are most difficult to measure that matter most of all.

"There's a need for new metrics that go beyond the traditional industrial-age measures that focus on cost analysis and savings," says Erik Brynjolfsson, an associate professor of IT at MIT's Sloan School of Business, a visiting associate professor at Stanford Graduate School of Business, and a leading proponent of the new ROI thinking.

Specifically, these ROI measures focus on things such as product quality off the assembly line, customer satisfaction after an interaction, and faster time to market. These factors, say new-age ROI thinkers, reflect a company's real sources of value and are what customers truly care about. Once these factors are isolated and measured, the proponents add, companies can better assess how IT supports them. "Technology by itself doesn't do anything," says Robert Benson, a professor of information management at Washington University in St. Louis. "It's what the organization does to use information and reach out to customers that matters. The purpose of IT is to change the behavior of its users to better achieve their business objectives."

Because this approach to ROI is so new, there are no measures of how many companies actually use it as the basis for their IT-buying decisions. But there's no doubting that some CIOs are already giving the "intangibles" theory a shot. "We're making IT i nvestments with the expectation that there will be a payoff in customer satisfaction," says Gene Trudell, general manager of computer services for U.S. Steel Group, a Pittsburgh unit of USX Corp.

Still, proponents of intangible ROI are far from unanimous in their thinking. One sticking point is how far ROI measures can extend. Some IT managers take a midway approach, factoring in intangibles but still demanding evidence that these assets result in dollars-and-cents gains. "We account for intangibles, but only to the extent that those things can be translated into quantifiable earnings streams," says James Hatch, CIO of Case Corp., a Racine, Wis., equipment maker. But others say many intangible benefits of IT must remain intangible. "It's hard to measure things like how important knowledge is to the organization," says Robert Walker, CIO at Hewlett-Packard. "What's the value of knowledge and information?"

These new ways of thinking about technology's ROI represent a clean break from the past, mainly be cause it now seems that past measures missed some of IT's biggest benefits. Instead, past approaches mainly looked for cost savings. Was a client-server system cheaper to install and run than a mainframe-and-terminals setup? If so, went the thinking, go for it.

That kind of thinking is still prevalent among IT managers. In a June InformationWeek survey of 100 IS managers, 45% of those polled said their organization's senior management requires that the IS group calculate ROI or some other type of economic measure on major technology projects. Nearly 80% of those surveyed said measuring ROI of technology projects is useful. Yet only one in five respondents said their organizations have formal ROI measurements in place for technology projects, and fewer than one in four have plans to adopt such measures in the next 12 months.

Another approach to measuring ROI was to compare technological speeds and feeds, using price/performance and other measures. But this "technology for technology's sake" thinking ignored the business context the systems were supposed to support. "Far too often, IT is not measured in business terms, but in technology terms," says Jerry Luftman, director of the information-management executive program at Stevens Institute of Technology in Hoboken, N.J. "Many IT people talk about Mips installed, response times, and systems availability-but not the kind of things businesspeople care about."

Yet another aspect of intangible ROI: The idea that when to buy technology is just as important as deciding which technology to buy. Brynjolfsson and others espouse the real-options theory of evaluating and analyzing IT investments.

Real options take into account factors such as timing of a project launch, factory expansion, or systems upgrade. For example, it might make more sense to wait six months to invest in product X than to buy a less-advanced product Y in three months, or to defer a major plant overhaul until market conditions warrant. "In some cases, it may be beneficial to delay inve stments to take advantage of and preserve your options," Brynjolfsson says. "Traditional accounting measures fail to take into account the value of options."

In practice, even some proponents of the intangibles theory use a mix of different ROI practices. Trudell of U.S. Steel, for example, sometimes uses absolute cost savings to justify an acquisition. Other times, he uses cost savings plus intangibles such as workflow improvements. "Measuring ROI is not the same as building a new test or doing something mechanical where you can measure the effectiveness of it," he says. "We don't have a hard-and-fast way to measure that so we can say yes or no on a project."

Still, Trudell says he expects that all new IT systems will improve customer satisfaction-and that satisfied customers will place more timely orders with U.S. Steel. For example, the company has a project in the works to install an integrated order-entry fulfillment system based on Oracle database applications, as well as a plan to replace lega cy applications.

The company wants to integrate data from the front end to the back end so receivables, invoicing, and order entry will all be integrated from the beginning of the process through to billing the customer. "Hopefully, that will allow us to better serve our customers by speeding the process of getting their orders into the system and consequently reducing overall cycle time," Trudell says. "Then they will be more willing to enter orders closer to when they need them."

Another intangible measure of ROI is shareholder value. "If you can demonstrate that a series of computer products is contributing positively to shareholder value, then you can rest easy at night that you're not doing badly," says Rick Smith, a partner with Giga Information Group, a Cambridge, Mass., consulting firm.

Competing with new thinking about the intangible effects of IT, several other new approaches to ROI are vying for the acceptance of CIOs. One approach is risk analysis. Oft-cited findings by the Standish Group, a Boston research firm, show that nearly a third of application-development projects will be canceled before completion. Of those that are completed, more than half will overrun initial cost estimates, and the average cost overrun is 189%.

"IT investments are among the riskiest a business can make," says Douglas Hubbard, director of applied information economics at DHS and Associates, an IT consulting firm in Chicago. Hubbard recommends that IS chiefs use a method called modern portfolio theory (MPT), which financial managers have used for years to optimize investment portfolios on a risk-and-return basis. "A large portion of the IT portfolio actually is far riskier than even a risk-tolerant investor would want to invest; it may rank up there with junk bonds."

Another important approach to ROI is called economic value added (EVA). Case CIO Hatch defines it as "cash-adjusted operating profit minus the cost of capital used to produce earnings." Key to making this work, says Hatch, is close work with the business units. "Our clients must take ownership on the benefits side of the equation," he says, "while IS takes ownership on the cost side, and then we can use EVA on a project-by-project basis."

"I make sure costs of ownership as well as life-cycle costs are clearly articulated," Hatch adds. "When mixed with the clients' benefit statement, we will give our senior management an opportunity to look at something beyond normal accounting and treat development costs of new systems or projects as an investment rather than an expense." Case will start using this measure in 1998 for all new projects. "We see it as a replacement for ROI, which is limited to just the near-term investment," Hatch says.

Interested in adopting the "intangibles" approach to ROI? Realize now it's not going to be easy. One third of the IT managers surveyed by InformationWeek, asked to name the biggest challenges to measuring the ROI of IT projects, cited the difficulty of measuring the true economic benefits of IT. More than one quarter said they can't determine an accurate accounting of IT returns, and almost as many said there's no comprehensive, reliable metric available.

Also, the old ways of measuring ROI still work sometimes. When HP's Walker switched from X.25 to TCP/IP as the protocol for the company's global WAN in 1989, "we saw a dramatic reduction in our absolute costs," he says. "That was a case where we made a very modest investment for a substantial return." The same was true when HP eliminated the last of its mainframes: The returns-in direct cost savings-were measurable within a year and a half, Walker says. When HP moved to a common operating environment for its PCs, managing them from servers rather than individually, the company saved a couple of thousand dollars per PC-or about $200 million a year, according to HP estimates. "That was done with a relatively modest investment," Walker says.

Others, among them some of the biggest names in business, question whether IT's value can truly be measured . Take Walter Shipley, chairman and CEO at Chase Manhattan Bank in New York. Asked how he justifies the money Chase spends on IT-nearly $2 billion a year-Shipley replies, "That's a good question, and a hard one to answer."

To be sure, Chase's board of directors approves the IT budget after examining project-by-project and business-by-business spending, then calculating rates of return. "But it's still difficult," says Shipley. "How do you determine how IT contributes to areas such as competitive differentiation? You may have an advantage over your competition in your capacity to mine data, but how do you measure that?"

Others insist the ideal ROI formula remains undiscovered. "The best minds in the world have worked on this for years and haven't really found an answer," says Jim Webber, president of Omicron, a 120-member IS management consortium in Mountain Lakes, N.J. He says the best approach many CIOs use is what he calls the "cumulative anecdotal evidence" method. Practitioners systematically foll ow up on projects after they've been launched to validate cost savings and other gains.

But considering intangibles may bring another benefit: more discussion between IT managers and their organizations' top executives. ROI measures that consider multiple factors "encourage management discussion about the value of IT and the cause-and-effect relationship between IT and the achievement of strategic business goals," adds Benson of Washington University. "I've never seen management convinced by dollar numbers alone. But when you demonstrate something like [improved] time to market or greater market share-things that everyone knows will add to the bottom line and increase shareholder value-then you connect with what's important to management," he explains.

Such buy-in from the top, of course, is crucial. "You must have a business buy-in for intranet ROI and for IT ROI in general," says Don Ryan, a VP of the Meta Group, a consulting firm in Stamford, Conn. A new Meta Group study-sponsored by IBM, Microsof t, and Novell-asked 50 IT managers to determine the hardware, software, and support costs of their intranets, as well as what benefits they hoped to accomplish, such as lower printing costs for publishing companies. Eighty percent of the managers said their companies generated a positive ROI for intranet apps, with an average annualized return of 38%.

Maybe those ROI intangibles aren't so intangible after all.

See related story: " ROI Profiles "

For one executive's views on the ROI debate, see related story: " IT Management ".

To see the full results of InformationWeek's survey on IT return on investment, check out techweb.cmp.com/iw/637/roi.htm

Back to News in Review

Send Us Your Feedback

Top of the Page