In 1989, with the first InformationWeek 500, we struggled to find the right formula to rank business technology users. The 500 biggest users had to be called out. But what about the most efficient ones? Or the most innovative and visionary?

"Big" was the operative word in the early years. We ranked companies based on the value of their installed base of computer and telecom equipment. That yielded a homogeneous list year to year, with American Express, AT&T, GE, GM, GTE, IBM, and Sears dominating the top of the rankings. This approach favored mainframe users as well as computer and software vendors that used a lot of IT for product development. Frustrated, we opted in 1993 and 1994 to analyze IT trends among the 500 largest publicly traded U.S. companies by revenue. This approach brought new names to the top 10--Exxon, Mobil, Philip Morris, and Wal-Mart--but not much variety.

In 1995, we returned to tracking business technology usage but implemented a more extensive research survey and weighting system that factored in the type of IT and how it was used. This approach brought in a new community of still big, but more cutting-edge, business technology users: AlliedSignal, Columbia/HCA, Gannett, International Paper, Kmart, and Lockheed Martin were regulars at the top of the list for the next few years.

Since then, we've been reevaluating and tweaking our criteria each year. In 1998, we landed on a questionnaire much like the one we use today that rewards the most innovative IT users based on quantitative and qualitative measures. Size still matters--only companies with $500 million or more in revenue can apply. But those that do qualify must prove that they're spending IT dollars in innovative ways. In the last few years, we've added essays: one on applicants' most innovative IT initiative in the last year, and the other on innovation in a specific technology area.

Since 1998, companies in the top 10 are a constantly changing group. Several have made it to the top 10 two years during that time, but only three--Avnet, Cisco, and E.&J. Gallo--have been in the top 10 three times.

Bottom line: InformationWeek 500 companies are big, but they need to bring it when it comes to innovation and execution.

Click here to see 20 years of InformationWeek