What To Test When You Can't Test Everything

By Ian S. Hayes

f you can't test everything for year 2000 compliance, how do you decide what to test? Risk-based testing is a method for focusing your testing efforts on areas that do the most to reduce the risk to the enterprise. You assign testing resources based on two key factors: the importance of a function, and the likelihood that a function will fail.

The importance of a function is defined by its business impact. You don't want to spend $50,000 in testing efforts to prevent a $1,000 problem. For instance, a year 2000 failure that causes the missorting of records in a back- office report is less important than a failure that causes a piece of hospital life-support equipment to fail.

But even if the consequences of a failure are high, the likelihood of that failure's occurring may be low. For instance, a financial-services firm's systems may fail to handle a specific securities transaction if the transaction occurs at exactly 12 a.m. on Jan. 1, 2000. That error could cost the company $10 million. But if the chance of that happening is one in a million, you may be better off fixing and testing a problem that has a $50,000 impact and a 50% chance of occurring.

The likelihood that an application will fail is related to the number of dates it contains and the number of changes that must be made to those dates. Programmers typically make three mistakes per 100 changes, regardless of the type of application. Based on those error rates, a 250,000-line application might require 7,500 changes for year 2000 compliance-resulting in about 250 errors that must be caught during testing.

With risk-based testing, you use these two factors to creat e your testing plan. Critical functions with many 2000 changes have the highest priority; less-critical functions with few changes fall to the bottom of the list. As you complete testing of the most crucial applications, the risk to the enterprise drops rapidly.

Risk-based testing is a compromise. The only way to eliminate year 2000 risk is to test all software and hardware components. But as year 2000 deadlines loom, compromise may be the only viable option.

Ian S. Hayes is a principal at Clarity Consulting Inc. in Marblehead, Mass.

Return to story " Testing For 2000 "

Back to News in Review

Send Us Your Feedback

Top of the Page