InformationWeek
Defining The Business Value Of Technology
November 17, 1997
Analyzing The Analysts
The vote's in: Analysts are vital, but could do moreBy Bob Violino and Rich Levin
| View survey results, "
How They Stack Up
" as a PDF file.
View the chart, " What The Analysts Offer " as a PDF file. SEE RELATED STORIES: The Crusader : Dale Kutnick, Meta Group The Up & Comer
: Gene DeRose,
The Patriarch
: Gideon Gartner,
The Golden Gut
: George Colony,
 To view a PDF file,
download the Adobe Acrobat Reader . |
echnology executives use the big IT analyst firms more than ever. But they still wish the analysts did a better job. These companies-Gartner Group, Meta Group, the Yankee Group, and others-have become the closest thing many senior technology managers have to strategic planning partners. "They're much more important to our business than they were five years ago," says Gene Trudell, general manager of computer services at U.S. Steel Group. "Technology is changing so rapidly, we can't keep up."
Adds Jim Hoyt, a technical consultant at Cigna Systems: "There are more choices to make and more interdependence among technologies. It's impossible to be an expert at everything."
Still, when IS executives consider the quality of the services and products they buy from the analysts, they see pl enty of room for improvement. In fact, the 10 largest IT analyst firms fared only so-so in InformationWeek 'ssurvey of 300 IS executives who use them. In all categories-including credibility, accuracy, customer service, strategic value, and business-strategy expertise-the mean grade (i.e., the sum of all the scores divided by the number of scores) given by IS executives to the analysts never topped 8 on a scale of 1 to 10, with 10 being the highest rating. In one crucial area-knowledge of the Internet-IS execs gave the analysts an average rating of only 6.7.
Even the analysts admit they can't be all things to all people. "All of us would like to be excellent at everything," says Howard Anderson, president of the Yankee Group in Boston, "but very few are."
In the same survey, when IS executives were asked why they have stopped using an IT analyst firm, the leading reasons were "poor value for price" and "missing technology trends."
Still, there's little question that IS executives use analysts more than ever. Gartner Group, by far the biggest and most dominant of the analysts, has seen its revenue grow by 75% in the last three years, to $395 million (this includes revenue from Gartner units Dataquest and Real Decisions Corp.). During the same period, Gartner's net income more than tripled, to $51 million. Similarly, Meta Group's revenue has nearly tripled in the last three years, to $37 million. And Forrester Research has seen its revenue more than double in the same period, to $25 million. (The other firms in our Top 10 list-Aberdeen Group, Giga Information Group, Hurwitz Group, International Data Corp., Jupiter Communications, the Yankee Group, and Zona Research-are either privately held or owned by other companies).
Analysts' services don't come cheap. Half the executives in the IW survey say they spend more than $40,000 a year on each consulting firm whose services they use. Many say they spend more than $75,000 a year on each firm they use.
IS departments also invest a lot of time reading analysts' reports, attending their meetings and conferences, and talking with them on the phone. On average, executives in the IW survey say they read three analysts' reports a month. Their organizations buy an average of 227 reports a year. Larger companies-those with sales of more than $5 billion-tend to buy substantially more market research. But IS executives at smaller companies-those with less than $500 million in sales-read an average of three reports a month, as many as executives at larger companies.
Nor are there any signs that the IT analysis business will slow, despite such apparent competition as free information on the World Wide Web. In fact, many businesses rely on the Internet for quick access to the analysts' fee-based services. Yet they still rely heavily on the experts to inter
pret information gleaned from the Web. "You get what you pay for on the Internet," says Cigna's Hoyt. "What's out there that's free is often unfocused and unreliable information. We use the analysts to save time and get information that's been digested."
IT vendors, whose products and services the analysts review and evaluate, also rely on the firms for inside information on their customers and markets. Hewlett-Packard's computer organization spends about $4 million a year on analysts' services to track multiple markets. "They can tell us what our customers really want," says Marsha Haugen, market-research manager for HP's computer group in Fort Collins, Colo.
A spokesman for Sun Microsystems, which also uses analysts, adds, "The analysts have their finger on the industry's pulse." Several other major IT vendors, including IBM and Dell Computer, refused to comment.
Studying The Competition
Vend
ors also use analysts to learn more about their competitors, and their competitors' customers. "What are people in Sun shops concerned about in terms of network management?" asks Haugen. "The analysts know those things." Adds Nancy Scull, director of analyst relations at Digital Equipment, "The analysts help us to see ourselves as others see us."
The roles played by the analysts can vary widely. For Jim Royston, analysts serve astrusted advisers. Royston is director of technology delivery at Eastman Kodak Co., which brought in both Gartner Group and Meta Group in 1995. At the time, Kodak was weighing a move of its 30,000 desktops from Windows 3.1 to Windows 95, Windows NT, or a combination of the two. Kodak knew the decision could help or hinder its IT efforts-and its overall business-for years, but the company's own managers couldn't agree on a decision.
Meta and Gartner analysts strongly recommended that the Rochester, N.Y., company move to NT, even though the platform was largely unproven at the time. Kodak took their advice, and today, with the project about halfway done, the company is glad it did. "The analysts helped us believe NT was going to be Microsoft's strategic direction," Royston says. "We had heard that from Microsoft, but sometimes you take what you hear from vendors with a grain of salt."
In fact, Royston says the decision to go with NT has saved Kodak both time and money. He's convinced that given Microsoft's support for NT, if Kodak had chosen differently, it would now be engaged in a costly double migration, first from Windows 3.x to Windows 95, then from Win95 to NT.
Some IS managers use the analysts to keep a lid on their own research costs and staffing. "We use them to supplement our internal expertise," says Al Spangenberg, VP of emerging technology at Prudential Insurance Co. in Roseland, N.J. "That way, we don't need a large consulting group on staff."
Still oth ers use analysts to help them negotiate contracts with vendors. U.S. Steel Group, a subsidiary of USX Corp. in Pittsburgh, brought in analysts when it wanted to purchase CMOS mainframes and negotiate Mips-based software pricing. The analysts suggested long-term licensing agreements that general manager Trudell expects will save the company many times the analysts' annual fees. "When you're talking about negotiating with a vendor that you're going to pay potentially millions of dollars, you want to make the right decisions," he says.
But the analysts are not for everyone. Virtual Mortgage Network Inc., for example, doesn't use any analysts. "This may sound arrogant, but we have a better handle on our technology issues than most analyst firms," says Phillip Lindsay, director of software engineering for the Newport Beach, Calif., company. "We hire the brightest people we can find, from executive management to engineering staff. Our corporate resumés show experience beyond most, if not all, analysts I've met, heard, or read."
Some companies that use the analysts nonetheless complain that they're too slow to react. "New suppliers are starting up all the time, and we need to find information quickly," says Monte Seifers, corporate contract manager for telecom equipment maker Northern Telecom Ltd. "We generally are asked to make a `go/no-go' decision on a supplier or a product within days." Still, Seifers admits that Nortel spends more than six figures annually on outside research.
Others rely on the hundreds of IT trade magazines, newspapers, newsletters, Web sites, and other sources of information that have emerged in the past two decades. Some even turn to their peers for advice. "Most of the technical information I need comes from CIOs at other companies," says Niraj Patel, CIO at GMAC Mortgage Corp. in Horsham, Pa.
Paul Stanton, director of marketing for Microsoft's enterprise customer unit, agrees that managers need to rely less on analysts and more on their own expertise and that of fellow IS prof essionals. "They need to share their best practices," he says. "Imagine if physicians went to industry analysts to learn how to conduct their profession."
The most important reason for using analysts, say executives in the IW survey, is to get validation for one's technology strategies (see chart, right). That was the case at insurer Cigna Corp. in Philadelphia, which asked Gartner analysts for their opinions on a broad plan to upgrade server and desktop operating systems, application development tools, and executive information systems. "Gartner used their insights from working with other customers and their knowledge of the industry to give us advice on the near-term future for some of these products," says Hoyt, who evaluates analysts for Cigna. "That brought an outside credibility and perspective to the product selection process. You don't want to make those decisions in a vacuum."
The next most impo rtant reasons for using analysts, according to the IW survey: vendor strategy evaluation and best practices. Other top reasons cited: forecasting new products and scoping out IT management trends.
( continued... )
