NIST Updates Software Testing Tool - InformationWeek
Government // Enterprise Architecture
12:13 PM
Ransomware: Latest Developments & How to Defend Against Them
Nov 01, 2017
Ransomware is one of the fastest growing types of malware, and new breeds that escalate quickly ar ...Read More>>

NIST Updates Software Testing Tool

Bug-detecting Advanced Combinatorial Testing System gets an improved user interface and methods to eliminate unnecessary tests, says National Institute of Standards and Technology.

Best Government Websites
(click image for larger view)
Best Government Websites

The National Institute of Standards and Technology (NIST) has updated a two-year-old software bug-detection tool with a new user interface and a new feature to improve testing efficiency.

The latest release of the institute's Advanced Combinatorial Testing System (ACTS) -- which uses a NIST-developed approach to testing -- includes a new feature that specifies relationships between parameters being tested. The feature can help testers eliminate unnecessary evaluation of possible code interactions that aren't valid, such as using Microsoft's Internet Explorer on a Linux system, according to NIST.

The release also includes a new tutorial for using the tool called Practical Combination Testing, a method developed by NIST researchers that ACTS employs. The testing method checks for bugs caused by the interaction of between two and six variables.

The tutorial includes concepts for using software tools to generate combinational tests and was written so even undergraduates in computer science or engineering can understand it, according to the institute.

Developed in 2008 by NIST researchers and available for free, ACTS takes a unique approach to bug testing by leveraging NIST research that found nearly all software failures are caused by only six or less variables.

Much of the time, software testing is done by checking combinations of input actions in pairs, and by testing as many of those pairs as possible. However, researchers at NIST said this isn't practical because it's time consuming and also could itself crash an application.

NIST researcher Rick Kuhn, working with others from the institute, found that between 70% and 95% of bugs are the result of only two variables interacting, and nearly 100% are triggered by no more than six. Kuhn worked with researcher Jeff Yu Lei and students from the University of Texas at Arlington to develop ACTS based on this concept.

NIST provided an example to illustrate the difference between traditional bug testing and ACTS. Testing all possible interactions for a product with 34 on and off switches would require 17 billion tests, according to the institute. However, ACTS can test all three-way interactions using only 33 tests, and all six-way combinations with just 522 tests, numbers that are far lower than 17 billion.

Since 2008, NIST has delivered ACTS to 465 organizations and individuals – including academics and governments. In addition to IT organizations – which comprise about half of ACTS users – the tool also is popular in the financial, defense, and telecommunications sectors, according to the institute.

NIST currently is working with Lockheed Martin to study how the company uses ACTS to test its software applications and will publish the results when research is complete.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
8/10/2012 | 6:00:54 AM
re: NIST Updates Software Testing Tool
excellent to know that Billion test cases can be covered by 32 tests. what is the confidence interval level for 100% coverage using this process. Would like to know moe details of the model as a professional
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of IT Report
In today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll