Amazon Web Services CTO Werner Vogels defended Amazon's bid for a CIA contract even though a Government Accountability Office (GAO) study found that AWS's bid was $54 million higher than a competitor's. That competitor was IBM, and IBM protested the CIA's decision last February, leading to the GAO review.

Being the high bidder is a new role for Amazon. In the past, it's boasted that the track record of its parent company, Amazon.com, as a low-margin retailer gave it the culture needed to make it the best purveyor of low-cost cloud services.

At last week's Structure event, GigaOm researcher Jo Maitland expressed amazement at Amazon's ability to outbid IBM. "It's fascinating to see the Amazon thing was more expensive than IBM. I never imagined I'd see that ... Hey, the CIA is willing to pay more for Amazon. Good for you, man."

Vogels didn't exactly know how to receive the congratulations. He chuckled and, after a pause, said, "It's an ongoing process. I think we're really focused on building absolutely the best technology that the customer wants. It's a long process." It took "many months" to understand the CIA system's requirements and formulate a response, he said.

"They picked Amazon’s because we could get the job done for them with superior technology than what they are able to get anywhere else," Vogels concluded.

[ Want to learn more about Amazon's low-margin approach to cloud services? See Amazon To Cloud Rivals: Try To Catch Us. ]

The CIA is looking for a private cloud, built to its specifications on government premises, but operated by a public cloud provider, that is capable of analyzing 100 terabytes of raw data on a cluster at a time. The CIA cloud will employ a MapReduce-based system to spread big data loads out over multiple clusters for simultaneous processing in the CIA cloud. MapReduce came out of a Google approach to mapping data out to a server cluster, then assigning analysis work based on which processor was closest to the relevant data. It's a way to use commodity servers with limited storage attached in a large-scale and high-speed manner.

A GAO analysis, while recognizing IBM had submitted the low bid, said that Amazon's experience as a public cloud provider made it the more likely candidate to produce a cloud that could scale up as needed.

"While IBM's proposal offered an evaluated price advantage over five years, the (CIA) concluded that this advantage was offset by Amazon's superior technical solution," the GAO ruling said. The CIA in evaluating bids had rated IBM's proposed risk as "high" and AWS as "low." It called the need to automatically scale up analysis applications the Achilles heel of IBM's proposed solution, calling that a "grave" concern.

In its protest of the decision, IBM called that part of the CIA decision "irrational."

IBM has experience providing cloud services, but not as much as Amazon. It has supplied platform-as-a-service to developers for several years and has offered infrastructure-as-a-service based on its SmartCloud Enterprise since April 2011. It said in March that SmartCloud in the near future would be based on the OpenStack project's open source code. On June 4, it announced it was buying SoftLayer Technologies, the largest cloud supplier still privately held in the U.S.

Somewhat in contrast to those recent moves, Amazon has been offering its IaaS since 2006 and won the credibility contest when it came to guaranteeing an ability to scale up.

That's a significant gain for Amazon, if the CIA's decision holds. The CIA has 60 days to confirm or change its contract award after the issuance of the GAO report on June 6. The contents of that report were first reported by Federal Computer Week June 14.

Several government agencies, including NASA, the GAO, the U.S. Army, and the Recovery Accountability and Transparency Board, which powers the Recovery.gov stimulus-tracking website, already use Amazon's cloud services. More are expected to follow as they try to meet the requirements of the Federal Data Center Consolidation Initiative, launched by the Obama administration. It seeks to close 800 federal data centers by 2015.

The GAO finding may prompt the CIA to consider rebidding the private cloud contract. It hasn't announced that it will do so, but the GAO review raised enough issues that it is likely to be rebid.

If Amazon beats out IBM for a four-year, $600 million CIA contract, many more federal agencies are likely to conclude they can't go wrong signing a deal with Amazon. The CIA appears to be trying to analyze data on a scale never attempted by the government before. If it establishes a successful private cloud in the first four years of the contract, it is expected to extend that contract out to 10 years. Such contracts could provide the guaranteed cash flow needed by an ambitious cloud builder to finance an expanding chain of data centers positioned to offer the best, low-latency services to its biggest customers.