The National Cancer Institute is reevaluating an expansive computing grid used for cancer research, following an assessment by its board of scientific advisers that the project is riddled with problems. NCI has invested more than $350 million in the project, called caBIG, including nearly $200 million during the past two years.
CaBIG, which stands for the Cancer Biomedical Informatics Grid, grew out of a pilot project launched in 2004 whose goal was to provide shared computing for biomedical research and to develop software tools and standard formats for information exchange. NCI-funded cancer centers across the U.S., including Mayo Clinic Cancer Center and Sloan Kettering Institute for Cancer Research, have access to caBIG via 145 network nodes. CaBIG comprises nine "workspaces," which support work in life sciences research, clinical trials, tissue banks and pathology, and other areas.
Following his appointment in July, NCI director Harold Varmus initiated a four-month review of caBIG, and that review was completed in February by a working group of the Institute's board of scientific advisers. The working group, in a report issued last month, found that fewer than a dozen cancer centers are using caBIG's clinical data management tools or its cloud computing infrastructure. And while acknowledging that caBIG has had a "positive influence" on creating standards for data exchange, the working group criticized the program for straying from its original goals and for sprawling into an overly complex "software enterprise" of more than 70 applications.
CaBIG is administered by NCI's Center for Biomedical Informatics and IT. The 10-member working group, comprising experts with advanced degrees in science and medicine, concluded that caBIG's program management structure has become bureaucratic and suffers from "very high" overhead costs. The program grew too rapidly without careful prioritization or a cost-effective business model, according to the working group. Lead contractors are Booz Allen Hamilton, SAIC-Frederick, and Sapient.
The NCI board issued 10 recommendations for corrective action. They include an immediate moratorium on all caBIG software development, including projects underway through commercial contracts; a one-year moratorium on all new projects and contracts; an audit of caBIG expenditures to date; and the creation of an independent oversight committee.
The working group's report details a long list of problems with the program. Some caBIG applications cost millions of dollars to build. The price tag for one, caArray, an array data management system, was $9.2 million. Other software is overdesigned, difficult to use, or lacking support and documentation. For these and other reasons, a majority of 32 "bench to bench" life sciences research tools developed for caBIG have had limited use or impact, according to the report.
Only seven of the 51 NIC-funded cancer centers participating in caBIG use the program's cloud computing infrastructure, caGrid. Twenty others are in the process of accessing caGrid, but that brings the total to just over half of those centers. Among the issues cited as obstacles are that caGrid is too complex and hasn't been adequately tested for security. Even the University of Chicago, which helped develop caGrid, isn't using it.
"If there was ever an example of how not to run a program, this may be it," says Michael Biddick, president and CTO of Fusion PPT, a systems integrator that works with government agencies. Biddick is an InformationWeek contributing editor who has written about caBIG. "The failure to link the mission objectives to the technology shows how important user acceptance and buy-in can be," he says.
Efforts to build a user community around caBIG, which include strategic planning meetings and monthly conference calls, were also found to be lacking. An annual meeting intended to advance the program was described in the report by one unidentified attendee as a "contractor feeding frenzy." In addition to the three prime contractors, about 20 other vendors are involved with caBIG.
The NCI report describes caBIG's program administration as having a "very complex organizational structure" and says that information on the program's budget has been inconsistent and difficult to decipher. The working group determined that money paid to Booz-Allen Hamilton, SAIC-Frederick, and Sapient over the past seven years exceeds $60 million and is "probably significantly more." It estimates total spending on caBIG to be at least $350 million for fiscal years 2004 to 2010, including about $100 million in federal stimulus funds.
