InformationWeek
Defining The Business Value Of Technology
January 18, 1999
 Print this story |
continued...page 2 of 3
Cost Xpert doesn't account for the time needed to put an application into production, which is another common deficit in IT project estimation. This is a huge cause of frustration to businesspeople to whom no one has mentioned implementation.
Cost Xpert doesn't allot a sufficient percent of the Lifecycle variable to testing, allowing only 39% for creating a test strategy, test cases, and unit, system, and integration testing. Best practice indicates that upwards of 60% of Y2K time may be spent in testing. However, Cost Xpert should get credit for including time for developing a test strategy and test cases. It does not appear to include acceptance testing, which can also be a major process.
Most of the deficiencies in Cost Xpert's estimating models can be overcome because Marotz has made most of the product, including the Lifecycle variable, customizable. However, since Marotz claims such estimating accuracy for its included methods, any adjustments might erode the accuracy of the results.
Cost Xpert's Risks variables for a year 2000 project don't seem to make sense. For example, low user satisfaction is unlikely because, in theory, no system changes should be happening. Inadequate user documentation is unlikely to be a problem because you are fixing the system's internals; you are more likely to suffer from inadequate or nonexistent system documentation.
Marotz doesn't tell you a lot about the Risk variables (the chapter in the user manual is only three pages) and suggests you get its Risk Expert product. However, since you don't know whether or how the estimating algorithms in Cost Xpert use the risk factors, this leaves you rather ignorant.
Cost Xpert uses seven different sizing approaches: five metrics, including lines of code and function points, and the two rule-of-thumb methods that are most often used in IT departments, and depend on rough estimates and rough comparisons with other, similar projects.
Cost Xpert lets you use more than one sizing approach at the same time. However, what good is using two or more of the estimating methods if the results can vary as widely as the manual suggests when it says, "Look for the estimated values to be within roughly 25% of each other"? On a big project, a 25% difference can translate into months of effort.
Marotz says Cost Xpert produces estimates accurate to plus or minus 5%. However, the user manual says what you get is the "mean [sic] schedule and cost," which "implies that half the jobs will exceed this amount and half will be completed in less than this amount." You can tinker with the estimate produced by Cost Xpert, which is useful if you think you know your environment better. But if Marotz has confidence in its product, it shouldn't say that what you get has only a 50% confidence factor.
While this is a cost-estimating package, most managers don't think in terms of dollars but in people's time. It would be helpful, therefore, if the user manual included both a dollar and a time measure for project size when guiding you toward which metrics to use.
Cost Xpert allows a number of changes to the Constraints variable that supports what appears to be a more realistic estimate than the one produced automatically. These changes included review time, activity overlap, risk tolerance, and contingency.
After my adjustments, Cost Xpert produced an estimate of 129.2 worker-months and an "optimal delivery time" of 11.9 months.
The work breakdown structure--the list of the project's tasks--included items specific to program management of a large project such as a Y2K fix, for example, for a systems inventory. The list of documents to be delivered by the project also included some of relevance to year 2000 program management. The labor estimate, on the other hand, seems to come directly from the effort to fix a 700,000-line program and doesn't include any program-management tasks. Cost Xpert understands the structure of a year 2000 project, including program management, but makes it difficult to understand how to estimate one.
The reports were adequate, and there is an ad hoc report writer. Some odd things appeared, though: reports that show information different from that on the related screen, for example.
Cost Xpert is probably best used as a supplement to other forms of project estimating for large projects, because of its concentration on dollars. On the other hand, for smaller projects it provides a quick and simple way of getting a work breakdown with costs thrown in.
KnowledgePlan 3.0 (beta)
KnowledgePlan from Software Productivity Research is a general project-estimating tool that includes some ability to estimate costs. It produces more detailed work breakdowns than Cost Xpert and has better integration with external project-management tools. KnowledgePlan provides custom import-export tables that replace the standard ones that come with Microsoft Project. You can merge data from KnowledgePlan with an existing Project file, and create a project in KnowledgePlan to use in estimating by importing data from a Project file.
KnowledgePlan uses a "knowledge base" of more than 8,000 projects. The manual says, "KnowledgePlan estimates are often within 5% of the project's actual costs and schedule."
If you need more-specific project comparisons, you can buy other, extra-cost knowledge bases from Software Productivity. Also, new to version 3.0, you can use the extra-cost "knowledge base calibration feature" to create knowledge bases.
KnowledgePlan has much less information about project estimating than Cost Xpert, although you can learn a lot about how to estimate simply by setting up a project and refining the estimate.
Installation was easy, and created about 41 Mbytes of program files.
KnowledgePlan uses tab sets for entering variables. Creating a project profile is simple and can be done with a wizard. KnowledgePlan also lets you estimate systems such as process control, robotics, communications, and others that are not traditional business applications.
There were no obvious bugs, even though this is a pre-release beta.
continued...page 3
return to page 1
Back to Labs
Send Us Your Feedback
Top of the Page
