CIOs and senior IT leadership teams often find themselves drowning in a sea of data, yet they seldom have the information they need to effectively optimize their organizations. This question was asked at a recent Six Sigma in IT conference: "What should we be measuring for our IT organization?" It typifies the data-saturated information shortage in many IT organizations.

The sheer volume of data available, coupled with the critical nature of effective decision-making by senior leadership teams, demands that large IT organizations declare the IT measurement program as a core competency. This requires allocation of sufficient horsepower (number of resources and adequate skills) to position the measurement program for success. A part-time commitment seldom yields a trustworthy repository of information that CIOs and their direct reports can leverage to guide the ship.

The quickest way to identify the IT organization's critical information needs is ask to the CIO: "What keeps you up at night?" While specific items for the IT measurement program aren't likely to be returned, some of the critical information needs will be identified: "Is our software quality getting better over time?" "Should we focus on reducing our system 'abnormal ends' any further?" "What is our backlog of high-severity security exposures?" "Can we prove any of this to our customers?" "What do our customers really think about us?"

The biggest critical success factor is to make sure there's support from the top. Grassroots measurement movements are a noble endeavor but are often destined for trouble when they attempt to move beyond a single functional area and represent the enterprise. Support from the top is needed to commission a specific person or team to define the enterprise standards and views.

The senior leadership team of Sallie Mae IT, for example, has established its support for the IT measurement program by making the collective success or failure of the measures included in the IT scorecard an integrated part of total compensation.