When Gen. Michael Hayden faced Congress last week for a pre-confirmation grilling as President Bush's nominee to lead the Central Intelligence Agency, he started by calling intelligence gathering "the football in American political discourse" since the 9/11 terrorist attacks. Then, when pressed about a megadatabase of phone records of U.S. citizens allegedly compiled under his watch at the National Security Agency, Hayden punted.

The nominee declined to discuss the sensitive issue in open session or otherwise address the wide--and sometimes wild--speculation about how much phone data the feds have collected and what they're doing with it. The issue got new life in a May 11 story in USA Today, which reported that AT&T, BellSouth, and Verizon had turned over phone call records of tens of millions of Americans to populate the NSA database. The purpose of the data collection, according to USA Today, is to identify potential terrorist activity. But privacy advocates teed off on how such a database might be misused.

The technology certainly exists to assemble a massive phone-records database, but it's not clear whether the NSA has the volume of data it would need to get a complete picture of terrorist activity or the data mining algorithms necessary to tell the difference between calls among friends and those among terrorists. Businesses and government agencies routinely mine multiterabyte databases to create meaning out of minutia. But the stealth nature of the terrorism business would make connecting the dots infinitely harder.

Indeed, it's unclear just what data the NSA has in hand. BellSouth and Verizon both denied sharing phone records in bulk with the intelligence agency; AT&T was sketchy about its participation; and Bush was noncommittal on whether such a database even exists.

Here's what we do know: The NSA is a sophisticated user of database technology--Larry Ellison has long said the NSA is one of Oracle's earliest customers. We also know that government agencies are intensely interested in data mining. A 2004 survey by the Government Accountability Office found that federal agencies were engaged in or planning 199 data mining projects, including 122 involving personal data. A database of phone records wouldn't be hard to create; the data exists.

We also know that terrorists make phone calls. After the 2001 attacks, the government determined that the 19 terrorists had made 206 international calls from the United States, according to press reports. A logical step for data analysts would be to search through phone records to see if there are other networks of people whose calls followed similar patterns.

Social Connections

In data mining, the practice of looking for underlying connections between people is called social network analysis. Phone data is useful because it helps expose relationships and associations among different groups. With social network analysis, contacts are commonly laid out graphically to illustrate connections and find patterns. At the simplest level, this could be shown as links similar to the spokes of a wheel leading to one source, indicating that a person holds a leadership position within a terrorist cell. Looking deeper, it could uncover relationships, such as two suspected terrorists linked only through a third, unknown person.

Valdis Krebs, founder of social networking analysis company OrgNet.com, conducted his own analysis of the 9/11 terrorists by collecting information from press reports such as who called whom, the addresses shared by the terrorists and their known associates, and information that they used the same frequent flier number. Krebs found that more links led to the group's leader, Mohammad Atta, than to any other terrorist.

Social network analysis could seek to create a "map" that shows characteristics unique among terrorist networks. When a credit card is used to make a very small purchase followed not much later by an attempt to make a large one, banks recognize that pattern as a characteristic of fraud. The challenge for the NSA and other intelligence agencies is to find enough unique characteristics to differentiate terrorist social networks from those of nonterrorists. "There are no clear-cut definitions for what defines terrorist behaviors," says Chris Westphal, CEO of Visual Analytics, which does data analysis for intelligence and law enforcement agencies. Financial transactions, wire transfers, border crossings, public records, travel records, and phone records might all be used to inform the process.

It's unknown to what extent the government may be using social network analysis, but the feds are clearly using data mining to fight terrorism. Fourteen of the 199 data mining projects revealed by the GAO in 2004 involved analyzing intelligence and detecting terrorist activities. Eight of those programs involved private-sector data, including personal information provided by data aggregators.