What determines where a vendor lands in Gartner's Quadrant? Gartner says its assessments are based on a vendor's viability, service and support, features and functionality, and technology. Among the criteria that gets factored in: input from customers or prospects who have piloted a product or chosen an alternative; customer experience with scalability and cost of deployment; a vendor's "commitment" in the form of R&D; marketing clarity and sales channel; a vendor's ability to support its vision; and overall corporate viability.

By comparison, Forrester's Wave assesses competitors in a given product category. Each report charts vendor performance on a grid featuring a vertical measurement of a product's strength and horizontal measurement of a company's strategy. Vendors are labeled leaders, strong performers, contenders, or risky bets, and their placement is marked with a dot, the size of which is determined by the vendor's "market presence."

All things considered, the methodology behind Forrester's Wave report is more transparent than Gartner's Quadrant, says Sandi Meyer, senior analyst-relations manager with Trend Micro. That worked to Trend Micro's advantage last month when Forrester listed the vendor as a leader in enterprise anti-spyware. Trend Micro has been a Forrester client since May. The company also subscribes to research from Canalys.com, Gartner, and IDC.

Lessons Learned

Research firm executives are well aware of the questions being raised about their business models, but don't expect changes to be fast or wide-sweeping. The financial stakes are too high--and the incentives for change aren't compelling enough.

Meantime, those vendor-sponsored research reports keep rolling off the presses and into the hands of business-technology buyers. Cyveillance, a provider of online risk monitoring and management software, funded an IDC report published in January titled, "Who's Got Your Virtual Back? Mitigating Online Security Risks." The goal was to provide market education, not to serve as a promo for Cyveillance, VP of marketing Todd Bransford says, so the only place the vendor's name is mentioned is on the cover. "We talked with IDC in order for them to validate what we're doing in the market," Bransford says.

Cyveillance isn't a regular IDC customer. Still, the vendor won't say how much it spent on the IDC report. In advance of publication, Cyveillance had input into the issues that should be covered, and after it was completed, a chance to review it. Cyveillance made no changes to the report's contents before it was released, Bransford says, but it could have requested them, and that's troubling enough. The IDC report preceded the release of Cyveillance's Intelligence Center 3.0 risk-monitoring software by only a few weeks.

Technology vendors often will sign up for analyst firm services when they feel their market is poised for growth. Cyveillance in January signed on with Gartner to help understand how its software fits with the growing demand for IT security and regulatory compliance. "I also wanted to understand how a product we're rolling out should be priced," Bransford says. So IDC helps seed the market, Gartner helps price it, and both get paid for doing so. Is everyone comfortable with that?