The findings, in a new report by the Government Accountability Office, reinforce an assortment of anecdotal evidence that some data is more thorough than others and that different agencies measure their IT progress differently. Although GAO noted that the IT Dashboard has had made "significant improvements to oversight and transparency," of the IT projects posted on the IT Dashboard that the GAO studied in-depth, half had "notable discrepancies" in cost and schedule data.
For example, the IT Dashboard shows the Department of Justice's Law Enforcement Wireless Communication as tracking within 5% of cost estimates between July 2009 and January 2010, but the GAO found that the project was actually off by more like 10 to 15% of estimated costs.
In a statement put out to accompany the report, Sens. Thomas Carper, D-Del., and Susan Collins, R-Maine, pointed out that the IT Dashboard shows a Department of Homeland Security investment to improve scanning of cargo at the border as on track and under budget, but an earlier GAO report found it to be costing $740 million more than planned.
In addition, although federal CIO Vivek Kundra has said the IT Dashboard was intended partially to help agencies manage their IT investments, officials at three of five agencies whose projects were reviewed by the GAO said that they weren't using the site for that purpose because they use alternative tools to do so.
For its part, the Office of Management and Budget, which manages the IT Dashboard, isn't just sitting back. IT Dashboard 2.0, which OMB launched last week, incorporates recommendations from both Congress and the GAO, according to an OMB spokeswoman.
For example, as part of the new release of the IT Dashboard, OMB improved its cost and schedule calculations by more heavily weighting more current data than historical milestones. "We do not rule out the use of additional adjustments to the formula as we assess these calculations in the future," federal CIO Vivek Kundra said in written comments to the GAO.
However, the site's trouble's go deeper than user interface problems, into the quality of the data itself. In his letter, Kundra admits these problems, referring to "known limits of the available data." Some of this problem is being reconciled by reporting requirements that began this year and by guidance on standardized investment activity reporting being developed by an inter-agency working group.
Clearly, despite its data problems, the IT Dashboard has come with some benefit, including improving data quality over what existed before. "Only after the IT Dashboard began using milestone data did the need to collect activity start dates and separate percentage complete fields become apparent," Kundra wrote to the GAO, noting that today the IT Dashboard includes start dates for every investment.
From another angle, the federal CIO's office uses the data to help conduct in-depth reviews of troubled IT projects called TechStat sessions and, Kundra has noted in the past, the IT Dashboard has in several cases created dialogue between agency secretaries and the CIO where there previously had been none.
Good data, though, not just improved data, is essential, according to Collins. "Much is at stake here," she said in a statement. "Aggressive oversight of the nearly $80 billion in taxpayer money that the federal government spends on IT investments annually is critical to helping ensure the prevention of waste, fraud and abuse.
