FBI's Sentinel Project: 5 Lessons Learned

American Red Cross Social Media Command Center

After six years of development, the FBI says its next-gen digital case management system, Sentinel, is finally up and running. FBI agents can now use the system to manage records electronically, with document templates, drop-down menus, and many other PC-like features.

Sentinel had been a case study in federal IT projects gone awry--missed deadlines, budget overruns, feature shortcomings, and a benchmark test last October that pooped out. The FBI Inspector General, in a 2010 report on Sentinel, cited "significant issues and concerns." FBI director Robert Mueller faced a grilling from Congress on how, when, and at what cost this all-important project would be completed.

Keep in mind that Sentinel has roots in an earlier IT project failure, the so-called Virtual Case File system. The agency pulled the plug on that effort in 2005 after pouring $170 million into it. So this week's announcement that Sentinel, as of July 1, became available to all FBI employees is a major achievement. Mueller, in a written statement, called it "an important step forward" for the FBI.

Whether the system will work as advertised and be accepted by the agency's rank and file remains to be seen. I've been following the Sentinel project closely for the past few years. Here are five takeaways.

1. Private sector expertise is valuable. The first step in Sentinel's turnaround was the recruitment of a private sector IT executive, Chad Fulgham, to oversee it. Mueller brought in Fulgham, a former senior VP of IT with brokerage firm Lehman Bros., as CIO in December 2008. Mueller said Fulgham's business experience would "fit well" with the FBI's needs. It wasn't long before Fulgham hired Jeff Johnson, also a former Lehman Bros. technologist, who is now the FBI's CTO. (Fulgham left the FBI in April 2012. More on that below.)

2. Agile development gets things done. The next big shift in strategy was Fulgham's decision in September 2010 to wrest control of the project from prime contractor Lockheed Martin and use agile development to accelerate software deliverables. The thinking was that a hands-on, incremental approach would be faster because functionality would be developed, and adjustments made, in two-week "sprints." The FBI missed its target date for finishing that work--September 2011--but it credits the agile methodology with ultimately getting the job done.

3. Commercial software plays an important role. Sentinel is based in part on commercial software, a fact that's often overlooked because of all the custom coding and systems integration involved. Under the hood are EMC's Documentum document management software, Oracle databases, IBM's WebSphere middleware, Microsoft's SharePoint, and Entrust's PKI technology. Critics who say that Sentinel would have gone more smoothly if only it had been based on off-the-shelf software seem unaware that, in fact, it is.

4. Agile development is cheaper, too. Sentinel came in under its $451 million budget. The caveat is that the FBI's original cost estimate for Sentinel was $425 million, but that was before Fulgham and Johnson took over, and they stayed within the budget they were given. The Inspector General might quibble with how the FBI accounts for the total project cost, having pointed out in the past that its tally didn't reflect the agency's staff costs. But the FBI wasn't forced to go to Congress with its hand out. Agile development wasn't only faster, but also cheaper.

5. Don't deploy new software on old hardware. The FBI learned that lesson the hard way in October when the system, during a four-hour test involving 743 users, suffered two outages. The agency made the mistake of running the test on legacy hardware, which it was forced to upgrade prior to a broader rollout. That caused a delay just as the IT team was approaching the finish line.

Johnson offered the first public demonstration of Sentinel this week at FBI headquarters. (Here's my review of the system from a demo I was given a few months earlier.)

Fulgham wasn't around to participate in the unveiling. He left the FBI this spring to take a job with Lockheed Martin--yes, the same company he elbowed aside in 2010 when choosing the agile development path. In his new role, Fulgham is a VP with Lockheed Martin's information systems and global solutions division, where he works with U.S. and international defense agencies.

In a phone call this week, Fulgham said he knew he would take heat for going to work for the same company that, as the FBI's CIO, he forced into a backup role on Sentinel. But he insists that Lockheed continued to be a key partner on the project, as well as on other FBI initiatives, such as its Next Generation Identification system.

Sentinel is "arguably the most important application at the FBI," and agile development turned out to be the right way to complete it, Fulgham told me this week. There's a lot of talk about how federal agencies can benefit from the best IT practices of the private sector. Sentinel's turnaround is the latest chapter in that play book.

The Office of Management and Budget demands that federal agencies tap into a more efficient IT delivery model. The new Shared Services Mandate issue of InformationWeek Government explains how they're doing it. Also in this issue: Uncle Sam should develop an IT savings dashboard that shows the returns on its multibillion-dollar IT investment. (Free registration required.)