FBI IT: Lessons To Live By

After more than three months chasing this week's cover story on the FBI's IT woes, I read with great interest a similar account in yesterday's Washington Post. The main difference between our stories is the Post's ability to get their hands on a confidential report to the House Appropriations Committee that blasted the FBI's decision to continue spending money on its Titanic-like Virtual Case File system, even after it was clearly destined to sink into the abyss. Both stories, however, reveal several uncomfortable truths about IT management.From my reporting and from my read of the Post, it's clear that the VCF was a project that suffered from both inertia and hubris -- tragic downfalls that aren't exclusive to the public sector. Whether you're CIO of a Fortune 500 company or a large government agency, it's never easy to admit a mistake and cut your losses, particularly when this undermines a large investment in time and money.

Let's look at some of the things that went wrong with the VCF along the way. First, you cannot discount the impact that 9/11 had on the VCF. The Virtual Case File system was originally conceived (prior to that horrible day) as a Web interface into the bureau's legacy systems. Once 9/11 laid bare all of the government's data-sharing deficiencies, the FBI smartly recognized an opportunity to fix its part of the problem. The problem was in the execution of that fix.

The crucial mistake at this juncture was deciding to custom-build the VCF into a case-management system. Although it's likely that there was no commercial, off-the-shelf software available for the FBI to buy that would have accomplished all of the VCF's objectives, the decision to build a unique system proved expensive and unwieldy.

Enter Science Applications International Corp., the company hired in 2001 to build the VCF. I decided I would be very skeptical of SAIC's claims when I interviewed them for my story. It seemed like a situation where they could use the press to blame the FBI for all of the VCF's problems. Yet SAIC made a compelling case to me when they talked about how they weren't given proper guidance during the system's development and how they were bombarded with change requests that hindered progress (they told me that an average of 1.5 changes per day were requested by the FBI over an 18-month period between November 2002 and March 2004). Strangely, neither the FBI nor anyone else I interviewed ever disputed SAIC's claims.

It seemed curious to me that, even though the FBI had decided the VCF would never see the light of day, they decided to run a test pilot from January through March anyway. What I didn't realize, and what the Post reported yesterday, was that the VCF pilot test cost taxpayers another $17 million. Talk about not being able to cut your losses. The overall project cost $170 million, with $104.5 million of that being written off as a loss.

Another problem with the VCF was IT management churn. The bureau had five different chief information officers between November 2001 and December 2003, when they hired Zalmai Azmi as acting CIO. This lack of longevity surely contributed to the VCF project's lack of consistency.

A final point I'll make is that the FBI had decided early on to do a "big-bang" cutover in bringing the VCF to life. This meant the system's usability wasn't checked against real-world criteria as it was developed. As such, the VCF became a phantom system rather than one that could incrementally help FBI agents and analysts better do their jobs.

My article this week goes well beyond the VCF to look at how the FBI is changing its ways. As best I can tell, the bureau is on track and already learning from the VCF's high-profile flameout. CIO Azmi is a man with a plan, and he's got the trust of the top G-man, FBI Director Robert Mueller. There's an enterprise architecture in place, and Azmi is in the process of taking complete stock of the FBI's IT operations.

The inevitable question of whether this is too little, too late will be answered in time. Until then, for the safety of our country, I'm sure we're all pulling for the FBI to succeed, no matter how many mistakes they've made along the way.