In a speech this week titled "The Economic Gains of Cloud Computing," federal CIO Vivek Kundra reiterated his contention that the U.S. government can get technology faster and at lower cost through cloud computing. Kundra's presentation coincided with the release of a report from the Brookings Institution which found that federal agencies can save 25% to 50% of IT costs by moving to the cloud, and, based on that assessment, that there's a "strong argument" for doing so.

My take: Not so fast. Some of the case studies being used to demonstrate the cost advantages of cloud computing are outdated, incomplete, or otherwise off the mark. Public sector CIOs (and their private sector counterparts) must do their own business-case analyses to determine where cloud services are in fact a cheaper alternative to do-it-yourself IT.

This question of cost savings is critical to government adoption of the cloud model. In a new, as-yet-unreleased survey of federal IT pros by InformationWeek Government, the No. 1 driver of cloud adoption, by a wide margin, is the expectation that cloud computing will lower the ongoing costs of IT operations.

Agency CIOs certainly want cloud computing to be cheaper. The federal IT budget for fiscal 2011 is essentially flat ($79 billion requested in FY11 compared to $81 billion enacted in FY10), which means they will need to lower costs in some areas to invest in others. Cloud computing is seen as a potential way of doing that.

In Brooking Institution's just-released "Saving Money Through Cloud Computing" report, author Darrell West pulls information from a variety of sources that reveal major cost savings in a handful of examples: 24% when the city of Los Angeles moved from Novell's GroupWise applications to Google's cloud apps; 40% when Carlsbad, Calif., chose Microsoft Exchange Online over GroupWise; and 75% when Miami, Fl., tapped Microsoft's Azure cloud for the city's 311 service.

Sounds impressive, but details on just how those saving are realized are fuzzy. In the LA example, West writes that the city -- had it continued to manage its own e-mail system -- would have spent $15,459,438 on "GroupWise e-mail licenses, upgrades, system applications, 90 file servers, and 13 staff positions," plus $7,536,804 on Microsoft Office licenses. But what's the split between software, hardware, and personnel costs in that $15 million? Was GroupWise at list price or a negotiated license fee? What are the unnamed "system applications," and how much do they cost?

Likewise, in the city of Miami example, where savings of 75% are attributed, vaguely, to "hardware, software, and staff efficiencies" in moving to Microsoft's Azure cloud services. That case study, by the way, came via Microsoft, not an independent observer.

NASA's Nebula cloud environment also serves as a point of reference in the Brookings Institution report, but with this caveat: "Specific cost savings are hard to compute because the project did not exist prior to Nebula." Taken together, these ambiguities provide a shaky foundation for Brookings' ballpark estimate of 25% to 50% savings in the cloud.

In his April 7 speech, Federal CIO Kundra said "many organizations" in the private sector are realizing "tremendous savings" through cloud computing. Yet, Kundra offered only one specific example -- the Department of Interior saving 66% by moving 80,000 e-mail accounts to the cloud -- and that, too, was lacking in the details needed to understand how such savings are derived.

Kundra and West are almost certainly right about their two main points: that cloud computing can lower IT costs and that government agencies should begin moving in this direction. But their evidence is unconvincing, highlighting a need for more detailed TCO and ROI studies.

Federal IT pros must stay open to the possibility that cloud computing won't always be the best answer. West points to a Booz Allen Hamilton report which found that many government data centers average only 12% utilization of their storage systems, and he suggests that cloud computing may be a route to better performance. Maybe so, but there are other, potentially cheaper, ways to improve storage utilization -- virtualization, thin provisioning, data de-duplication -- without having to re-architect the data center as a storage cloud or move to a public cloud service.

How do government agencies make the right decisions on which way to go? InformationWeek Government will soon launch a series of reports and Webcasts designed to help with the assessment of cloud computing. Part one of our series, a report on "The Business Case for Government Clouds," will be published early next month.

Meantime, look beyond the easy claims of cloud savings for hard evidence. If you've done your own cost comparisons, drop me a note at jpfoley@techweb.com. We could all use the reality check.

John Foley is editor of InformationWeek Government and his Government Technologist column is a regular feature.

InformationWeek has published an in-depth report on cloud computing and service-level agreements. Download the report here (registration required).