Government // Cybersecurity
Commentary
8/12/2014
09:06 AM
Connect Directly
RSS
E-Mail
50%
50%

Cyber Risk Dashboards: False Sense Of Control?

Federal programs promoting the use of risk dashboards can boost real-time visibility, but only if they are used correctly.

Dashboards are used throughout business and industry to provide a measure of success. A correctly designed and implemented dashboard can provide critical information to an organization about performance and risk measures in near-real time. The dashboard information should drive the organization to excel in meeting goals while minimizing risk and provide early warnings of possible problems. Dashboards are a good thing when used correctly, but how do we know if we are measuring the correct indicators?

As part of the Department of Homeland Security's Continuous Diagnostics and Mitigation (CDM) program, organizations must develop a dashboard to allow DHS to assess the level of risk by agency. DHS states it will do this by collecting input from sensors placed in 144 agencies. These sensors will allow an agency to "quickly identify which network problems to fix first, and empower technical managers to prioritize and mitigate risks on their respective networks." While this goal is admirable, we need to examine the methodology proposed to ensure the dashboard will function as intended.

[Make sure you can bounce back fast. Read Cyber Attacks Happen: Build Resilient Systems.]

At a very basic level, risk is a function of the likely impact a threat could impart by exploiting a vulnerability. If a dashboard is to accurately reflect the true risk posture of an organization, it must take into account all aspects of the risk equation including threat, vulnerability, likelihood, and impact. The following diagram from NIST illustrates the relationship between the components of risk and also helps explain how we should arrive at organizational risk:

So how well does CDM capture organizational risk? Presently, CDM focuses on four main control areas: hardware assets, software assets, configuration settings, and vulnerabilities. The second phase of CDM promises access control management, security-related behavior management, credentials and authentication management, privileges, and boundary protection. The final stage will provide event planning and response, generic audit/monitoring, document requirements, quality management, and risk management.

Given the structure of risk, agencies must determine if DHS CDM feeds provide a collectively exhaustive picture of threats, vulnerabilities, impacts, and likelihoods for their risk dashboards. If these feeds are inaccurate, not timely, or incomplete, the dashboard risks will be inaccurate or worse -- divert scarce resources to the wrong risk areas. Agencies must map sensor information and capabilities to ensure they are not only getting accurate, thorough, and timely risk information, but also ensure they are leveraging the considerable investment in deploying and operating the sensors.

For the most part, the first four areas -- hardware, software, configuration and vulnerability -- are focused on vulnerability detection. Vulnerability is a major component of the risk formula, but we would be wrong to equate vulnerability alone with risk because we have not considered threats, likelihood, or impact. Therefore, a vulnerability dashboard treated like a risk dashboard might result in spending resources on systems that might not need the most urgent patches or repairs.

For example, two identical systems might be running the same operating system on the same hardware with the same configuration. One system is used to store classified information, and the other simply processes publicly available information. If a vulnerability scan is executed, both machines will be subject to the same vulnerabilities, as they have the same configuration, hardware, and software. Should a vulnerability exist, determining which machine should be patched first is a challenge, as they both show the same level of vulnerability criticality. Worse, let's assume the machine with classified information has a moderate vulnerability, while the public machine contains a high vulnerability due to a configuration difference. In this situation, if vulnerability is equated with risk, the machine with public information will be prioritized higher than the system with classified information.

Although this is a very simple example, it shows how distorting one area of the risk equation for another can cause disastrous results. When we don't accurately measure and combine the elements of risk, we might gain a false sense of control in our environments and be caught off guard when a breach, outage, or compromise occurs. Agencies must use existing information such as their FIPS-199 categorizations for impact and threat information from their security operations centers or intelligence programs to help bolster their real-time risk scoring dashboards. Combining disparate sources from existing C&A work in addition to often organic and nebulous threat information will prove to be the ultimate challenge for an accurate risk dashboard.

In the meantime, agencies should use the dashboards with a full understanding of what they provide -- and, more importantly, what they do not.

NIST’s cyber-security framework gives critical-infrastructure operators a new tool to assess readiness. But will operators put this voluntary framework to work? Read the Protecting Critical Infrastructure issue of InformationWeek Government today.

The authors are members of the (ISC)2 U.S. Government Advisory Board Executive Writers Bureau, which includes federal IT security experts from government and industry. The experts write anonymously through the Bureau so they can be more forthcoming with their analysis and ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
MrJackBadger
100%
0%
MrJackBadger,
User Rank: Apprentice
8/12/2014 | 5:17:19 PM
Awesome!
Totally concur with the article.  Vulnerability does NOT equal risk!  Everyone loves pretty dashboards though and so that's what we are trying to produce.  Instead of the "three ring binder" fiasco of the C&A soon it will be the "the dashboard said I was green and I got pwned!"
DanWaddellCISSP
50%
50%
DanWaddellCISSP,
User Rank: Apprentice
8/12/2014 | 10:00:48 AM
Great article
Emphasizes the need to make sure we are putting accurate data in these dashboards.  Otherwise, dashboards can do more harm that good.  Dashboards need to instill a sense of trust right from the get-go, or else the decision makers will ignore them.  If you can't measure it, you can't manage it.
Cyber Security Standards for Major Infrastructure
Cyber Security Standards for Major Infrastructure
The Presidential Executive Order from February established a framework and clear set of security standards to be applied across critical infrastructure. Now the real work begins.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Must Reads Oct. 21, 2014
InformationWeek's new Must Reads is a compendium of our best recent coverage of digital strategy. Learn why you should learn to embrace DevOps, how to avoid roadblocks for digital projects, what the five steps to API management are, and more.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
A roundup of the top stories and community news at InformationWeek.com.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.