InformationWeek.com

Welcome Guest. | Log In| Register | Membership Benefits

January 18, 1999

To Err Is Human, To Estimate, Divine

Many factors lead to poor project delivery; two tools can help developers improve value by turning informal record-keeping into a managed process

By Phillip Gordon

oor project estimation is the dirty little secret of application development. Nearly all projects are either canceled or come in over budget or behind schedule. A recent study of 100 companies found the average company completes only 37% of major IT projects on time, while only 42% finish on budget. The Standish Group International, whose 1995 Chaos Report highlighted IT project failures, found that 40% of application development projects are canceled before completion. Standish Group calculated these failed and struggling projects cost U.S. companies and government agencies $145 billion annually.

Vendors have addressed this chronic miasma with project-estimation software. I looked at two tools for project estimating and planning: Cost Xpert 2.0 from Marotz Inc. and KnowledgePlan from Software Productivity Research.
IT managers have had plenty of advice on how to improve this dismal state of affairs. The Software Engineering Institute has a Capability Maturity Model with a five-step ladder to perfection. Interestingly, the largest number of groups with the highest CMM rating are in India, now a world center for outsourcing of application development.
Another important method is the System Development Life Cycle (SDLC), a set of best practices that lets developers break projects down into recognizable and repeatable steps, processes, tasks, and outcomes, each of which can be accurately estimated.
One can argue there are other significant factors that lead to bad project delivery, most of them political. While this may be true, you can control your performance and that of your staff far better than you can control organizational politics. That makes it worth the effort to improve your ability to estimate projects. It's worth overcoming the political pressure to "just start coding," at least for those shops that have ever been over budget or past due on deliverables.

Dipping Your Toe In
You can start with something as simple as keeping a repository of project information, work breakdowns from your SDLC, estimates, and actual time spent by task. Such a simple approach can give you an informative history so that each new project builds on knowledge gained from the past. In my experience, one benefit of keeping this kind of record was the realization that only about 60% of a programmer's time was available for real work. The rest was consumed by sick days, vacation, holidays, training, meetings, and administrative functions, such as filling out time sheets so we could track projects. Knowing this allowed far more realistic estimates.

Some software claims to help turn the act of managing development projects into a science. This software, at the least, can help you turn informal record-keeping into a managed process. It can add further value if it helps guide you through the estimating process by culling from best practices and industry research, and if it provides rules of thumb for estimating.

There is one vital question to answer before choosing this software: Can you, a project manager, use it without taking a class, hiring a consultant, learning a new terminology, or adjusting your entire approach? You should get the benefits of better estimating without adding a huge overhead. Therefore, there are important criteria in evaluating such software:

It should have a simple set of easy-to-follow instructions that let you enter some information and get a well-organized and immediately applicable result. If you have to spend days gathering information, working abstruse formulas, or doing any other elaborate preparation, then stick with a pencil, an eraser, and the back of an envelope.
It should permit you to export the results into a file format read by your project-management software so you can track actuals against estimates. If not, it should import project-management files' actuals into the product and compare them. If you can't do either of these, you'll have to reenter the data.
It should use a well-known framework or model for estimation. Better yet, it should use at least several of them for comparison and contrast. Best of all would be compatibility with one or more standard SDLC and its estimating model.
It should allow you to estimate a smaller project (lower limit: three worker-months) very quickly, a midsize project (one to two worker-years) in a few hours, and a big project (for example, a year 2000 effort) in no more than a day, with some reasonable prework.
It should let you produce estimates using multiple dimensions: by team, person, or phase. A useful bonus is the ability to include required hardware and software in the estimate to get a complete "executive summary" of costs and time.

In addition to these must-haves, there are several nice-to-haves:

Idiot-proof installation, compatibility with Windows 95 or 98, no hidden extras to buy and install, no extraordinary reconfiguration to do, and a standard file system for backing up the estimates.
A clean, uncluttered user interface that follows standard Windows conventions. It should deliver clear error messages in standard English, and should recover gracefully from both your mistakes and the program's burps.
The ability to create a good-looking presentation, with the estimates, without having to retype data.

I tested the two products against these criteria using an actual year 2000 project from a billion-dollar company as the test vehicle.

Cost Xpert 2.0 (beta)
Marotz's Cost Xpert is a cost-estimating tool. It also produces some work-breakdown structures that you can exchange with Microsoft Project, although the process is rather cumbersome.

Marotz claims Cost Xpert produces cost estimates that are within plus or minus 5%, "given accurate inputs," and within plus or minus 2% of "other commercial estimating products costing $50,000 to $200,000."

Cost Xpert comes with a lot of useful information about project estimating, including a package of articles and studies on project planning, cost estimating, risk planning, and defect prediction.

Installation was a breeze, placing about 18 Mbytes of files on the hard drive. Cost Xpert doesn't make changes to configuration files, or modify .ini files. To uninstall, you delete the directories it creates.

Cost Xpert comes with a sample project, and the examples throughout the manual refer to it. Such consistency, added to the ease of installation, shows a well-thought-out product.

Cost Xpert uses the standard gray background tab set for entering variables. Creating a project profile is a simple process, mostly involving check boxes and a small amount of data entry.

There are some small bugs, not surprising since this is a beta, but otherwise the software is generally clean. For example, a number of fields, such as "company name," won't accept characters to their fully displayed length.

The product had an immediate problem with the test scenario: It was not readily apparent how you could use Cost Xpert for a large, complex, multisystem project such as Y2K changes. For example, it forces you to choose a primary and secondary programming language when creating a Y2K project. On a Y2K project, you are working on your entire application inventory, which might include six or more programming languages. It is possible to enter multiple modules to represent the systems in the inventory, but instead I used a single large application (700,000 lines of code) to create an estimate. It doesn't approximate the level of effort in a Y2K project, but stands as a useful proxy for a similarly large, complex project.