New Security Imperative: Demonstrating Results

As an IT or security manager, you have good reason to feel you've been rolling with a lot of punches lately. But there's another one coming, and it will be critical to your perceived value to your organization.

IT professionals will soon be challenged to prove, in measurable ways, the value of their information security efforts. Dealing—even successfully—with crises, management plans, and budget requirements will no longer be enough. It's one thing to measure performance based on inputs or proxy indicators—it's different, and increasingly important, to measure performance based on impact and results.

Look, for example, at the work by the Office of Management and Budget and Congress' Government Reform Committee to drive and evaluate the progress of federal agencies to secure their systems. By all anecdotal evidence, these programs are producing results, but the stated assessments are based on inputs like managing resources consumed (such as money, staff hours, and software installed) or on indicators that are only proxies for measuring security (such as systems certified). Nowhere is there an attempt to directly answer the most critical question of all: "How much more secure are we now?"

Certainly, estimates of the financial impact of viruses and cyberattacks are being produced, by firms including Computer Economics and the Cyber Security Institute/FBI survey. But a study I recently conducted with graduate students at Carnegie Mellon University analyzed these results and found imprecision and questionable assumptions underlying some of these conclusions, to the point where most "estimates" of the cost of a cyber-security event appear to be wildly overstated.

So what? Why is accurately measuring performance going to emerge as the next major challenge for cybersecurity professionals?

Simply stated, cybersecurity hasn't received board-level attention, but that's beginning to change.

For many boards of directors (and also senior non-IT managers), cybersecurity has been viewed as one of the "black arts"; it's been the province not of mere mortals but of highly trained specialists who can converse in the arcane language of DES, SSL, and CVE (Common Vulnerabilities and Exposures). And, with the real costs of cyberintrusions poorly understood, many organizations have held the view that "we can eat the costs" of whatever damage comes from intrusions or abuses. The consequence is that, for most organizations I have dealt with, cybersecurity programs have rarely had to directly answer the question "Exactly how much more secure are we now than we were before?"

Here's a prediction: If you haven't already been asked this question by your board of directors (or its equivalent), you're very probably going to face it in the next 12 to 24 months. Several important changes will drive this profound shift in board-level thinking.

First, cybersecurity is now a much more prominent issue than it was a few years ago. The rapid expansion of the cybersecurity industry has a corollary: budgets for cybersecurity are increasing, both on an absolute basis and as a share of total corporate IT spending. The impact of viruses and increased news attention to cybercrime doesn't escape senior management's attention, nor does cybersecurity as a focus of both U.S. and global government policies. The message that security affects the bottom line is being received at the highest levels—loud and clear.

Even more important, however, is that the demands for demonstrable security performance are rapidly escalating. Domestically, regulations requiring specific cybersecurity performance targets are now affecting, or soon will affect, most U.S. companies. To protect customer privacy, regulators are interpreting the Financial Services Modernization Act (Gramm-Leach-Bliley) to require systems security for financial services providers, with specific standards on the way. The Sarbanes-Oxley Act, passed after the Enron meltdown with the idea of preventing falsification of financial information, also has focused attention on the security of data and systems. And Health Insurance Portability and Accountability Act regulations covering health-care providers have their own detailed and technical cybersecurity requirements.

Along with regulation comes growing liability issues. Significant fines—and adverse publicity—already have been assessed by the Federal Trade Commission against at least one global firm for inadvertently releasing customer-specific information on a Web site. Expect more to come. As interpretations of tort liability evolve, it's also likely that lawsuits will be filed, claiming damages because of poor cybersecurity practices on the part of defendants. While this hasn't yet happened (to my knowledge), the attention of the trial bar to this issue is large and growing.

Finally, as the cybersecurity insurance market—though currently still a boutique specialty—continues to grow, the demands to specify performance to exacting risk-management practices (just as fire codes have to be followed) will become important.

The net effect is that IT executives will be seeing more demands to specify and quantify not just efforts and actions, but performance.

So what can you do? Fortunately, there are some immediate actions you can take: Create an explicit record of cybersecurity incidents affecting your organization, and maintain it in a consistent fashion over time. Help in adopting a good incident taxonomy is available from several organizations including CERT/CC and SANS Institute—use it. Work with your chief financial officer to develop an explicit and precise methodology for estimating the cost of cybersecurity incidents affecting your organization. Don't rely on vague generalizations or estimates from the press. Question the work of the outside consultants who work for you. Unfortunately, there's no good common methodology for estimating economic impacts (the subject, by the way, of a future article), so the rule here is to be consistent and precise, and have the support of the financial staff in your work. As well as estimating the costs of incidents that have occurred, also develop an approach to estimate the costs avoided through good security. If rapid response prevented the spread of a virus, you've saved your organization a lot of money and effort. Again, working with the financial staff, develop a means to quantify these benefits. Try to benchmark your cybersecurity performance against outside measures. Unfortunately, there are no good, reliable statistics—even the best (from the CERT/CC) are based on voluntary reporting, and therefore lack statistical rigor. Other sources can include your industry's security organization, your risk advisors or insurance providers, and information sharing with other organizations like yours.

The key is to develop ways of demonstrating—specifically, quantifiably, and defensibly—your impact on your organization's cybersecurity.

Unfortunately, at least for now, little help is available from the federal government. There's no comprehensive database of cybercrime/cybersecurity incidents, and, despite recent changes in law, most organizations don't voluntarily report incidents or vulnerabilities.

Measuring the value of your security efforts isn't a simple proposition. But remember: It's going to be key to your success in the future.

Jeffrey Hunker, Ph.D., was senior director for critical infrastructure at the National Security Council, specializing in cybersecurity. He is principal of Jeffrey Hunker Associates, consulting with both the public and private sectors, and also is professor of technology and public policy at Carnegie Mellon University. His columns appear monthly on InformationWeek.com. He can be reached at [email protected] or through www.jeffreyhunker.com.

To discuss this column with other readers, please visit the Talk Shop.