Back in the 1990s, when IT budgets were particularly tight, testing tools were often the first victims, deemed unnecessary in the face of more pressing concerns. Often, IT manager dispensed with the use of automated testing tools when developers were running late, figuring that manual testing would suffice.
No longer. The software world is increasingly complex. Today's software is multitiered, distributed, and integrated, often involving next-generation processes such as Web services. In the past few years, a strong push to integrate Web-based technologies with traditional systems, such as enterprise resource planning (ERP), customer-relationship management (CRM), or back-end mainframe systems has increased the complexity even further.
Vendors catering to the software-testing market are incorporating more advanced features, and integrating various types of testing capabilities, to create more-coordinated suites of testing tools. Today's tools are more efficient, accurate, and thorough. They're also easier to use and can test a greater variety of functions than previous generations of testing tools.
These advances are responsible in large part for many companies' developing formal testing methodologies--something that was relatively rare several years ago. Witness financial services company H&R Block Inc. Although quality assurance has been part of a structured approach to software testing for the company for the past eight years, "it's when the processes began to mature and we began to learn more about quality assurance and the types of tools that were available that we became more interested and willing to invest in those types of tools," says Mike Deloney, director of quality assurance. Today, H&R Block uses tools from a variety of software testing companies, including Compuware Corp. and Empirix Inc.
Most experts agree that it's important to test applications for functionality, scalability, and performance at every step of the process: preproduction, production, and postproduction.
During preproduction, or development, there are three types of testing tools:
Developer-oriented tools. Also called component or unit testing, these tools address issues such as memory leaks and other early performance problems. These tools are used to test specific pieces of the developer's application code independent of other units of code within the same software application. They're especially useful in testing large applications where different developers are working on different parts of the application.
Functional testing. These tools, usually used by a company's quality-assurance department, test that the code being developed is acting as expected. All tools in this category have scripting capabilities, allowing developers to modify what they've captured to test additional items. One capability of a functional testing tool is capture/replay, which runs sample tests against a working version of a program, capturing the activity it generates. During the playback phase, developers can see whether they're getting the expected results. Another subset of functional testing tools is regression testing, which tests whether a specific function continues to work normally after other changes have been made to the code.
Load testing. Also called stress or performance testing, these tools test what happens to the code when multiple users--sometimes thousands or tens of thousands, depending on the application--access the application simultaneously. Load testing is more critical in a Web environment, because it can be difficult to predict the volatility of the load change. These tools will help drill down to lower levels of the code to find out where bottlenecks exist and what's causing delays.
Testing after the application goes live requires a different set of tools. These include:
Performance monitoring and management. These tools rely on a variety of metrics gathered by monitoring the use of the application. Tools in this category will allow a tester to run a script to determine the type of performance the application is providing at different points during the day. If performance falls below a defined threshold, the system can be set up to notify a developer, who can then find the problem and fix it. Monitoring in a production environment helps ensure that business requirements are continually being met.
Test management. This type of tool manages testing across the board, letting developers and quality-assurance managers track tests that are being run on various applications, trace those tests back to the original requirements, and determine what errors were found during those tests. These tools are also useful for determining whether changes need to be made to the testing tools themselves, given changes to the application.
Because postproduction testing, which involves debugging, monitoring, and analysis, is done after deployment, many don't consider it testing in the classical sense. These tools collect statistics and failure information while the application is in full deployment.