InformationWeek.com

Welcome Guest. | Log In| Register | Membership Benefits

October 25, 1999

Millennnium Crunch:
For Y2K, Trust Is Not Enough
Independent verification and validation is an effective tool for managing risk and uncertainty

By Leon A. Kappelman

Leon A. Kappelman 've gotten a lot of questions lately about independent verification and validation (IV&V) vis-�-vis Y2K. Here are some of the best questions, along with my answers.

An attorney client recently asked:
I have heard a variety of views on the IV&V issue. Some IT professionals have told me that they don't believe IV&V is necessary for Y2K because the people developing the software are the most qualified to test it. Others have expressed the view that it's important to have an independent test to confirm the developers' approach. What's your view?

IV&V is not a binary, yes-or-no issue. IV&V is a risk- and quality-management tool. Like insurance, it helps us manage risk and uncertainty. When the probability of a loss is too large or too uncertain, or when the consequences of a loss are too disturbing, we buy insurance to manage or control the risk. Likewise, when a system is important enough, we take measures to ensure its continuity, such as backing up data, developing disaster-recovery plans, creating fault-tolerant systems, and bringing in independent validators.

Risk propensity, both personal and business, is also a factor, as is the confidence one has in the technologies used or in those who did the work. Would you fly in an airplane piloted by your desktop operating system?

The notion of having developers be the only validators of their own work is absurd, for software or any other technology. Could a corporate controller tell the board of directors, or the board tell the Securities and Exchange Commission, that internal and external auditors were a waste of money since the controller had designed the books and was thus most qualified to check them? Would you buy a home that wasn't looked over by municipal building inspectors?

Another client writes:
We need a reputable consulting firm that can do IV&V on several software products. Could you recommend any?

Selecting the right provider depends on your situation, technically and in other ways. As with insurance or financial audits, there are degrees of IV&V. For instance, there are differences among a document check, a code audit, a validation of test procedures and results, and an actual test of the software.

There are many kinds of tests. Furthermore, there are the typical vendor-selection issues such as brand name, depth of pockets, kinds of assurances desired, size, speed, budget, timing, availability, languages, platforms, experience in your industry, personal relationships, and history. In short, I'll need more information to make a recommendation.

The following question came from a credit reporting company with more than 25,000 suppliers of data that cannot be validated by the company for date-processing accuracy:

We've gotten letters from our suppliers telling us everything is Y2K-happy, but testing their systems is not economically practical. What other information can we get to help us manage supplier risk and uncertainty?

While I may accept the verbal assurances of my auto dealer for a vehicle on warranty, that would not be sufficient from, say, a manufacturer of hazardous chemicals. So, first, try to categorize your suppliers by some measure of criticality, which in this case is the amount of data provided. Even in cases where the likelihood of problems is low, if the consequences are grave, you'll still want to dig deep for information and get answers to all of the following questions:

Terminology: How did they define "Y2K-whatever"? Such terms as "Y2K-ready," "Y2K-compliant," "mission-critical," and "tested" are often defined politically, not technically.

Testing: Which systems or processes were actually remediated, and which were not? Remember, percentage completed is a useful metric, but it provides only a partial picture of reality. Can we see the test plans and results? Did a third party audit or validate the test? Can we see the audit results?

Continuity: What parts of the enterprise (such as departments and levels of management) were involved in contingency planning? Were the contingency plans tested? How? Did a third party validate or audit the contingency plans? May we see the test and audit results? How long will your continuity service center be open, and who will be on duty?

Keep those cards, letters, and E-mails coming, folks.

Leon A. Kappelman is associate director of the Center for Quality and Productivity at the University of North Texas, and co-chair of the Society for Information Management's Year 2000 Working Group. Reach him at kapp@unt.edu.

Back to the Columnist page
Send Us Your Feedback
Top of the Page