InformationWeek
Defining The Business Value Of Technology
November 15, 1999
http://www.informationweek.com/761/analyst.htm
The Knowledge Merchants
There's a symbiotic relationship between analysts and IT execs that revolves around one word: Trust
By Bob Violino
echnology has a habit of turning conventional wisdom on its ear. Take that old saw about knowledge and getting ahead: In IT, it's not who you know, it's what you know--and that's the problem. "Anyone in IT who tells you they're completely up-to-date is naive," says Dennis Benner, VP and CIO at Fluor Corp., a construction and engineering company in Aliso Viejo, Calif.
This is true despite the fact that technology managers have never had more sources of information about their world--or maybe because of it. Many managers say the plethora of industry data--from sources such as trade organizations, business and IT periodicals, internal data-gathering systems, and the ever-expanding Internet--demands analysis and interpretation that's difficult and time-consuming to generate. This is especially true with such fast-emerging opportunities as E-commerce and E-business.
"It's easy to have a modest awareness or understanding of what's going on," says Benner. "But to have an in-depth, detailed understanding and to know what's important for your business involves more than just skimming press releases and articles."
That's why many companies depend on research and analysis firms to help them spot new products, figure new strategies, and sort real trends from false leads. These days, that imperative comes from the top.
E.P. Rogers, VP and CIO of Mony Group Inc., says that earlier this year, the company's CEO was on a panel at a West Coast conference with Larry DeBoever, at the time an analyst and now general manager of worldwide sales and marketing at Meta Group. "By coincidence, the two sat next to each other on the return flight, talking about technology the whole time," Rogers says.
When the New York insurance and financial-services company started planning its E-commerce strategy this summer, the CEO brought in DeBoever. "We had a great dialog," says Rogers. "Larry shared his experiences in this area, and we're now in the process of building our E-business. He was a catalyst for moving it forward."
That close association is increasingly characteristic of the relationship between analyst firms and their business clients. At their best, research firms provide insight and analysis of the IT industry's most current products, trends, and practices. They do this through a variety of means: by polling users of technology, tracking the shipment and sales of products, interviewing vendors, and analyzing news reports and industry bulletins. Also, analyst firms compete with each other--and, increasingly, with user companies--to hire the brightest and most experienced IT talent.
Companies are tapping a variety of services offered by research firms, according to a survey of 271 IT executives conducted recently by InformationWeek Research. The most popular of these are electronic newsletters, alerts, and bulletins--which makes sense, given the current frenzied business and technology climate. It also tracks closely with the results of a similar survey conducted two years ago. Also popular are research reports, either by single or multiple topics. Businesses, on average, buy more than 100 research reports from these companies annually, according to the survey. Larger companies (those with more than $500 million in annual revenue), are more likely than smaller ones to purchase these reports; they buy an average of 175 per year.
What are managers reading about most? No surprise: The Internet and how to develop E-business strategies are among the most popular reports. Three-quarters of the managers say they've bought Internet infrastructure reports, and about seven in 10 have bought E-business strategy studies during the past year. Other hot topics include business applications and operating systems.
Fluor's Benner says his company has come to rely on Meta Group's expertise for nearly all its major IT undertakings: "Whenever we're doing something that takes us into a technology with which we don't have strong expertise, the first questions I always ask people are, have you talked to Meta and what are they saying? And, is this a blind alley or something we can build on?" Benner says Fluor spends more than $100,000 annually on services from the firm. "We don't have them making decisions for us, but they get us a list of candidates--whether it's technologies or suppliers or service providers--to narrow our search. They have the expertise we don't."
Fluor is in the midst of a reorganization that includes the formation of a new policy governing IT spending and standards for every line of business in the company. Benner has relied heavily on a Meta analyst to guide him through the process based on similar work he'd done for other clients. The time it has taken to get a high-quality IT policy paper completed is dramatically less than if he'd used the traditional approach of drafting a document, then having everyone go through it and make changes, Benner says, adding that, "I'm doing in three days what would have taken more than a month."
Many executives, feeling the pressure to make crucial decisions more quickly than ever, say time savings is probably the biggest payback from using these firms' services. "The volume of work that needs to be done is too much for us to handle internally," says Mike O'Dell, information services director at Wacker Silicones Corp., an Adrian, Mich., chemicals company that used Gartner Group, the largest and best-known of the analyst firms, to help formulate its E-business strategies. "Gartner helps us to avoid products that are doomed to fail from the beginning. There's simply not enough time for us to do it ourselves."
NextLink Communications Inc., a Bellevue, Wash., communications company, relies on research by the analyst firms to confirm its own findings and when making decisions such as migrating to a new technology. Gartner's best-practices and trends data has allowed the company to benchmark itself against similar-sized companies in the same industry, says Michael Brekke, NextLink's LAN systems manager. Brekke says he doesn't make decisions based on the research alone, but considers it good supplemental information. "It's not the be-all and end-all, but it helps," he adds.
The leading analyst firms have become so influential that their opinions can help IT chiefs gain senior-management approval for technology investments. At G&T Industries Inc., a foam-manufacturing company in Grand Rapids, Mich., IT director Steve Springsdorf received the go-ahead on projects in part because Gartner research supported certain technologies. Senior management has "an awareness of the technology" because of the Gartner research, which Springsdorf says helps add credibility to the IT organization's decisions. Springsdorf most recently used Gartner research to help select an enterprise resource planning package. "We would have had to consider 100 packages," he says. "The research helped narrow our search down to a few vendors."
Many companies turned to analyst firms for help in solving the year 2000 problem. A large majority of the managers surveyed are pleased with the advice they got. Only 5% say that they weren't at all satisfied with Y2K analysis and research.
Some used the firms to help select effective Y2K tools and services in what rapidly became an overcrowded marketplace that included inexperienced vendors. Mony Group employed Meta to come up with a short list of reliable Y2K vendors. "If we had to do it ourselves, I'm certain the list would not have been as complete and the process would have taken a lot longer," says CIO Rogers.
United Parcel Service of America Inc. in Atlanta did all of its own Y2K conversion and testing, but brought in an analyst firm to evaluate the work. "They made sure we were on the right track compared with other companies," says Jerry Skaggs, VP of IS operations at UPS. "We used them as a sounding board and benchmarking tool."
IT executives say the most important reason to buy research reports is to validate their company's IT strategy (see Behind The Numbers). "We do our own research as far as implementing technologies or analyzing vendors," UPS's Skaggs says. "But we use these firms to see if we missed something." UPS has its own internal research groups that analyze and rank vendors in storage management, mainframes, midrange systems, desktops, and advanced technologies.
Other reasons for buying reports, according to the survey, include learning about best practices and understanding E-business. Managers also find them useful for detecting IT management trends, evaluating vendors, and market forecasting.
While bulletins and reports fill a need, some managers say they get their biggest benefit from consulting, either face-to-face or over the telephone. "The one-to-one meetings are the most important," says Fluor's Benner. "If I want a better understanding of storage area networks, I know that if I call Meta today, I'll hear back from them today or tomorrow and spend an hour on the phone with someone who's got good knowledge of storage area networks." He adds: "I could spend time going over research documents and reports to gain a similar level of understanding, but life's too short."
When asked to identify the most important criteria for selecting a research and analysis firm, survey respondents cite quality of advice and the firm's overall reputation. A relatively small number of managers give importance to such factors as accuracy of product forecasts, reliability and trust, and experience.
"The way we see it, the better the firm's reputation the more likely it is to be accurate and credible," says David Robbins, chief technology officer at Conning Corp., a St. Louis provider of asset-management services for insurance companies, which has used Forrester, Gartner, Giga, and Meta at various times for advice on Web activities, Y2K work, development of an IT architecture, and implementing new platforms.
Data quality and accuracy depend a lot on the source of that information. Most, if not all, analyst firms do research work for IT vendors, helping them plan product development and marketing strategies. Some even specialize in working with vendors. The analysts then use those close relationships with vendors to aid in the analysis they do for users. It's a complex, co-dependent relationship with significant ramifications. "Analysts need to know what we're doing, and we have a need to tell them," says Debra Wood, an analyst relations manager at Sun Microsystems.
Sun has research relationships in particular with Forrester, Gartner, Giga, and IDC, as well as smaller, vertical-market-oriented firms, Wood says. Sun buys research reports from the various analysts, and its product groups work with the firms in planning and developing products, she says.
Hewlett-Packard also uses the firms for help in developing and marketing products, says Bonnie Rogers, director of analyst relations. "I look at our relationship with them as a partnership," says Rogers. "They help drive our strategy and future plans. They also take a lot of our plans to the users." HP spends millions of dollars annually on its analyst relationships, Rogers says. "These relationships have always been extraordinarily important to us."
Compaq works with all of the major analyst firms, and even includes them on its advisory counsels to help with product analysis, says Ken Conway, manager of analyst relations. "We've made changes in our products based on analysis, particularly with our mobile products," Conway says. Generally, the firms "are fair in their assessment of our products and services," he says. "But we don't always agree with them."
Microsoft works with analyst firms to refine its product plans and marketing strategy based on extensive feedback from users, says Karan Khanna, a lead product manager in the company's business and enterprise division. "The individual product teams regularly engage analysts" to find out what customers want and what trends could have an impact on Microsoft product development, she says.
ERP software developer SAP relies on the firms to fine-tune its product marketing strategy, says Nicole Milstead, director of analyst relations. "We use them at the end of the product cycle, when we're ready to go to market," says Milstead. "We test the solution and the message out on the analyst community."
For example, SAP recently brought in AMR and Forrester to examine the marketing strategy for its mySAP.com Web portal. "We had been suffering from market confusion over mySAP.com, and the analysts helped us to understand the severity of that confusion," Milstead says.
Other major IT vendors--Dell, IBM, Oracle--were reluctant to comment on their relationships with analyst firms, other than to say that analysts provide useful information on marketplace trends and product demand.
Smaller IT suppliers are perhaps even more dependent on analyst firms. Made2Manage Systems Inc. in Indianapolis, which develops manufacturing applications, uses research from AMR and Gartner to help plan product offerings. "In 1997, Gartner predicted that customers wouldn't buy software products anymore, but would be leasing them," says a research analyst at the company. "It alerted us to the importance of Internet applications, and now we've made an effort to include that in our products."
FVC.com, a Santa Clara, Calif., maker of broadband interactive video networking products, relies on the analyst firms' ability to gather large volumes of data from both users and other vendors. "These analysts have long-term, ongoing relationships with both customers and vendors," says Mark Cowan, director of marketing at FVC.com. Cowan says analyst firms provide perspective from both sides of the fence: what vendors are telling analysts about what they're doing and planning to do, as well as analysis of what customers are buying. "As a vendor, you can't get the information you want from other vendors," he says.
Because of that co-dependent relationship with vendors, analyst firms need to be on guard about appearing to favor certain vendors, particularly those that are clients. IT managers give the major firms high scores in credibility, but at the same time take steps to ensure they're not affected by any lack of objectivity.
"You always have to protect yourself," says Robbins of Conning. "You have to do your own homework and due diligence. If you read enough of the work from the same analysts, you can start ferreting out the bias."
Some believe bias toward vendors is widespread in the analyst community, even among the major firms. "They're totally influenceable, and larger vendors have the most influence," says FVC.com's Cowan. "Some of them are quite literally in people's pockets and will say virtually whatever they're told to say. Others are more ethical and aren't manipulated."
A lack of thorough coverage, rather than favoritism, is the biggest shortcoming of many analyst firms, especially the largest ones, says Anthea Stratigos, president and co-founder of Outsell Inc., a Burlingame, Calif., company that follows the IT research and analysis market and provides advice on how to best use information content. Firms such as Gartner Group don't have a big-enough picture of the entire IT marketplace to represent it accurately to their clients, says Stratigos. "Many IT vendors don't have enough access to Gartner analysts to tell them what they're providing and to give them the information Gartner needs to be aware of what they're doing," she says. "I don't believe that this is something Gartner is doing intentionally, but Gartner needs to represent the vendor community more holistically."
SAP's Milstead also questions the ability of some analyst firms to cover the length and breadth of the enterprise systems market. "In some firms, you'll have one or two analysts who cover the whole front and back enterprise apps space," she says. "It's a huge challenge to know all the players and cover the all the ground. They don't have enough truly comprehensive coverage."
The coverage may not be comprehensive, but it's certainly relevant and meaningful, argues Michael Fleisher, president and CEO of Gartner Group. "Our analysts watch many radar screens in their research activities, and rarely miss the activities of vendors with relevance to our clients' IT strategies," says Fleisher, who was named to his posts at Gartner last month. "Our approach is to provide thorough coverage of vendors determined to be appropriate for our clients to consider in their decision making."
Fleisher says maintaining independence from vendors is something about which every firm in the industry must be continually vigilant. "We have one asset to sell: the integrity of our intellectual capital," says Fleisher. "We cover the marketplace the way we see it, regardless of whether a vendor is a client. Vendors are good clients, and we have good relationships with them, but no amount of money would be worth undermining our integrity."
Dale Kutnick, CEO of Meta Group, tells vendors before taking them on as clients: "We will try to help you get your strategy right, but when we talk about you, it does not mean we'll say good things about you." Kutnick says Compuware Inc., the $2 billion software tools vendor, was a big client of Meta's, representing about $120,000 worth of business. "We said some true but not positive things about Compuware, and now they won't even talk to us." Kutnick says Meta gets at least one letter a month from vendor clients that claim they were insulted and threaten to sue.
Still, the conflict-of-interest charge isn't totally unwarranted. Some executives at the analyst firms say there's a common perception among many IT vendors that they can buy coverage from the firms. "Vendors perceive they have to buy services to be covered, says Tony Friscia, president, CEO, and founder of AMR Research. "Then, when they buy services, they want to know why they're not covered."
Robert Weiler, who was named president and CEO of Giga Group in August, is the former senior VP of worldwide sales and marketing at Lotus Development Corp. Weiler says he often got analyst firms to publish favorable information about Lotus.
"If we wanted a white paper written, we knew which analysts to call," says Weiler. Also, Weiler says, he was personally involved in editing white papers published by analyst firms. "They'd say, 'we found three customers who don't like the product,' and we'd say 'well, you talked to the wrong ones.'"
Weiler declined to identify the analyst firms he dealt with while at Lotus. He says he looks at things differently now that he's with Giga, which has policies in place to maintain independence from the IT vendors it covers. "Vendor clients aren't thrilled with what we're writing, but they appreciate the objectivity," Weiler says.
Some IT managers have had mixed results from the work they've done with analyst firms. American Maplan Corp., a McPherson, Kan., manufacturer of barrels, screws, and other products, was pleased with the service it got when the company hired Ziff-Davis Market Intelligence (which recently sold its market-research organization to Harte-Hanks Inc.) to quickly get up to speed on computer-telephony software products. That resulted in the selection of a Java-based PBX that Ron Relph, head of MIS at the company, calls "the most forward-thinking direction we could take in computer telephony."
But Relph isn't happy with the advice his company got from Gartner Group in selecting an ERP system. The system Gartner recommended--which American Maplan purchased--doesn't include a relational database, something that turned out to be a critical element, Relph says. "In my book, we could have gotten a product 10 times better for one-tenth of the cost."
Larry Olson, former CIO of the state of Pennsylvania and now a principal at aligne Inc., an IT management-consulting firm in Philadelphia, characterizes his experience with the leading analyst firms while a CIO as "mixed at best." One firm, which he declined to identify, "didn't understand that we were the customer and that they needed to tell us how they would add value."
A representative from the firm asked for a substantial hike in price for a service--$900,000 compared with $190,000 it charged the year before--without detailing any new benefits. "Needless to say," adds Olson, "we didn't accept their proposal."
There are others who find services from the firms too costly. "We've looked at services offered by the analyst firms, but they're very pricey for what we felt we needed from them," says Larry Hazen, IS director at Granite Construction Inc. in Watsonville, Calif. "There's so much information available today from the Internet, the media, and other resources that's just as good."
About 30% of the survey respondents say their companies spend more than $75,000 on services each year, and 17% spend upwards of $105,000. Some companies shell out as much as $500,000 or more on research and analysis annually.
Another complaint is that analyst firms often don't provide enough detailed information in their research data about specific markets. "Industry analysts that do market analysis add up their numbers at such a high level that if you want to dissect them to find out about a target market, you often don't get a great deal," says Cowan of FVC.com. "They aren't good at documenting precisely what it is they're measuring."
For more granular data, Cowan turns to smaller research firms that focus on specialized markets, and are "much better able to respond to questions and back up data" than many of the large firms.
The big research firms argue that the depth of their analysis is one of their strengths. But they concede that one of the biggest challenges they face--one all too familiar to IT chiefs--is hiring and retaining experienced and talented people. "Finding great people, whether it's researchers, consultants, or sales representatives, is always tough," says Meta Group's Kutnick.
Some firms have a full-court press on in many areas of IT at once. "The hottest areas in research are supply-chain management, customer-relationship management, and newer E-commerce applications," says AMR Research's Friscia, who adds that AMR has significantly increased its staff during over the last three years to support these and other areas. In December, 1997, the firm had 50 people; by the end of this year it will have 160. "The majority of those are research analysts," Friscia says.
All of the firms are trying to boost their expertise in the area of research and analysis most on the minds of technology managers today: E-business. "It's where the largest demand is coming from," says Giga Group's Weiler, whose firm is trying to recruit former and current CIOs, among others, for its E-business practice. "We want people who know how to build a business around electronic supply-chain management, and how to connect legacy systems and ERP with front-end customer systems," Weiler says. In August, Giga Group hired Larry Paul, former CIO at Carrier Corp., who guided the company's E-business initiatives as director of its E-practices division.
IDC is adding about 60 analysts a year, most of whom have experience with Web technology and E-business planning, says president and CEO Kirk Campbell. Gartner Group, which launched an E-business research service in late July, is trying to recruit E-savvy analysts as well as retraining many of its current 800 analysts in E-business issues. "In the last 12 months, we've focused our analysts on understanding the needs of our clients in E-business," says Fleisher.
Forrester is looking for researchers who know about E-commerce issues and are also experts in specific industries, such as utilities, says Stuart Woodring, VP of research. "There's an unbelievable shortage of good people," he says. "Anybody who says they can hire all the people they need is lying."
Analyst firms are also developing products and strategies to cash in on the growth of E-commerce. Last week, Gartner made a 70% investment in cPulse, an Internet company that monitors and benchmarks customers' satisfaction with a company's Web sites. Gartner says it will make customer satisfaction data available to clients, in aggregated and anonymous form, to establish a "Web customer satisfaction" benchmark. Also last week, Giga unveiled the Web Site ScoreCard, a service that evaluates a company's Web site. The service provides a detailed Web-site analysis and compares the findings with competitive and related "best-of-breed" sites.
There are about 140 firms providing some type of IT research, and new ones, particularly on the Web, are emerging regularly. "They're popping up everywhere," says Stratigos of Outsell. "We're going to see a lot of new Internet-based research companies emerging."
More competition should mean lower prices, but it also reduces these services to commodity status. "The growing competition is diminishing the perceived value of the information," Stratigos says. Although IT managers still find research products and services important, the survey shows that they're buying about 10% fewer of these than they were two years ago.
When FVC.com deployed a Web-based sales and marketing automation application, it used information not from analyst firms but from product reviews in trade journals, says Cowan. "We examined a lot of reviews that gave us a short list of features and characteristics that were absolutely essential. It allowed us to weed out 80% of the vendors who probably weren't big enough as companies or mature enough in terms of product age or were just missing critical features."
Although Brekke of NextLink uses the Web and industry publications to gather information, he doesn't think the Internet will make research reports from Gartner Group or the other firms obsolete. "It's hard to ascertain how reputable information is on the Web, unless you can be sure the company providing the information is reputable," he says.
"There will always be a place for paid analysts," adds Rogers of Mony Group. "We're using the Internet to supplement the intelligence we get from Forrester and Meta, and it's hard to picture how we'd be able to get advice from the Web that's so targeted and specific."
Rather than being usurped by the Internet, analyst firms are moving aggressively to expand their use of the Web to deliver the latest information to clients. For instance, more than 90% of IDC's clients receive information from the company in some electronic format, says Campbell. The firm is expanding its online research databases and offering Web "tele-briefings" to meet the growing demand for fast information, he says. "Making information easier to access on the Web is a big challenge and a key for us," Campbell says. "People don't have much time. The goal is to make it as easy to use our site as it is to use the most popular consumer Web sites."
Indeed, about three-quarters of the managers surveyed by InformationWeek Research prefer to get reports either by E-mail or by downloading them from the research firms' Web sites, rather than by postal delivery or fax. "We can get reports from these companies instantaneously on the Web and do searches for something that happened nine months ago," says Rogers of Mony Group. "You can find the information in five seconds, rather than searching through three-ring binders in the file cabinet."
Says Forrester's Woodring, "The challenge for the research and analysis community is to figure out how to get past paper as the delivery medium and make it a high-value, personalized experience for our clients." To that end, Forrester in March introduced an interactive E-Research service that lets clients customize the firm's research in a variety of ways.
Still, despite the current infatuation with the Web, Gartner's Fleisher says his firm will continue to use every tool available to deliver service to its clients. "Some only want to use E-mail, others want to interact with our Web site, and still others prefer to call and ask us questions or meet with us face to face," Fleisher says. "You can't operate with a single delivery model--you have to be able to provide what the clients want."
Even though competition is growing, the big analyst firms continue to enjoy financial success. For example, Gartner recorded revenue of $734 million for the year ended Sept. 30, which represents growth of 18% over the previous fiscal year; net income rose 9% to $107 million. Giga's revenue last year soared 97%, to $39 million from $19.7 million in 1997. Forrester's revenue for the same period jumped 53% to $61.6 million, and Meta's revenue increased 42% to $72.8 million.
U.S. companies will spend a total of about $2.9 billion on IT research and analysis in 1999, according to Outsell. The eight firms detailed in the InformationWeek study will account for about 42% of that, or $1.2 billion, Outsell reports. Large companies are devoting an average of about 6% of their IT budgets on research and analysis services, or about $2.8 million a year, according to an Outsell study of 185 IT professionals.
IT executives say this is money well spent if it means saving resources and getting critical analysis and data in a timely manner. "The key value we get is that we don't waste time pursuing things that have a low probability of success," says Fluor's Benner. "That makes it hard to measure the benefits, since it's more an avoidance of wasted time and effort, and it's impossible to quantify that."
Rogers of Mony Group agrees. "Early on, when we first started using research firms and the cost came up on the budget, it was an issue with senior management," Rogers says. "But we've made the case that for us to replicate this expertise in-house in so many areas would cost us 10 times as much, and I'm not sure we could even do a good job then."
Analyst firms are being forced to respond even more quickly to client needs--a direct result of the frenetic pace of change in technology and the demands businesses are placing on IT. "CIOs need to make decisions more quickly, and that will drive the way we approach consulting," says Meta Group's DeBoever. "We're meeting with executives at Aetna now to talk about how to change their IT infrastructure. They want to do it in three days, not 12 months."
That frantic pace is what will keep the analyst firms in high demand, says Fluor's Benner. "These companies are much more important than they were several years ago, because of the speed of technology change," Benner says. "The only way we can successfully use IT to be competitive in our marketplace is to talk to someone who's making a career out of knowing what this is all about."
--with additional reporting by Marianne Kolbasuk McGee, Tischelle George, Aisha M. Williams, Alorie Gilbert, Ramin P. Jaleshgari, and Aaron Ricadela.
Photo of Rogers by Michael Greenlar
Visit us at: http://www.informationweek.com
Boeing seeking Software Engineer 5 in Anaheim, CA
KForce seeking Inside Sales Associate in San Diego, CA
Amalgamated Bank seeking Chief Information Officer in New York, NY
Apollo College seeking Medical Billing and Coding Instructors in Albuquerque, NM
Allstate seeking Exlusive Agent in Las Vegas, NV
For more great jobs, career-related news, features and services, please visit our Career Center.
