The Parasoft SOAPtest tool that ABN Amro is using measures three main areas of quality testing: functionality (how well the program works), load (how it performs under strain), and regression (whether any code changes result in problems). SOAPtest can evaluate both the performance of Soap transactions at the server level and the user experience at the client level. It does black-box testing, which compares the actual responses from a Web service to the desired responses, and white-box testing, which tests the internal construction of the components that provide Web services.

SOAPtest lets users be sure Soap characters are what they're expected to be, says ABN Amro's Raj.

Using SOAPtest, ABN Amro's engineers can study the functionality of a Soap envelope as it carries data from one source to another, ensuring that data doesn't get corrupted in the process. "SOAPtest is a very good and very fundamental tool," says Paul Raj, lead quality-assessment engineer at ABN Amro. "It's allowing you to intercept the Soap characters as they come along and examine them to make sure they're what you expect they'll be."

Parasoft, a privately held company founded in 1987, built its business on developing tools for testing C and C++, Java, and Web-development software before expanding into Web services. It's working closely with IBM to integrate Parasoft with IBM WebSphere and is developing relationships with other key vendors, Parasoft president Adam Kolawa says.

Many companies are buying software that tests for quality in code and applications, but its use isn't ubiquitous. An InformationWeek Research survey of 800 business-technology managers in April found that 60% of companies are using commercial testing software. Yet Kolawa, like Lanowitz, is convinced that number will grow with Web services. The beauty of Web services is that vendors and companies are developing applications that adhere to the agreed-upon open interfaces and standards, but there's still room for slight differences in interpretation that could create communication and performance problems, Kolawa says. "Web services will require higher-quality software, and the only way you can automate testing is through the tools that are offered."

While Web services can be subjected to load and performance testing, Giga Information Group analyst Uttam Narsu says that integration testing isn't always possible, because developers often lack access to the applications that the Web services they're developing will ultimately connect with. The best they can do is make sure the interface is working properly. "That interface is like a contract," he says. "Service availability is always an open question when you're talking about Web services."

Because Web services are such a new technology, testers also lack standards such as performance benchmarks for measuring things like quality-of-service levels, Narsu says. And just what needs to be tested with Web services will depend on the implementation. Load testing will be a priority for Web services open to potentially thousands of Internet users, for example, while it will be less urgent for business-to-business Web services with low transaction volumes.

Mercury Interactive, a nearly $400-million-a-year company and the largest of the software-testing vendors, is building Web-services support into its product line, rather than developing new products that target the area. In February, it updated LoadRunner, its software for testing systems under heavy use, and Astra LoadTest, for testing scalability and performance, to support Web-services applications built with Microsoft Visual Studio .Net, an XML-based development environment.

Mercury Interactive says a new product it released in July, called ProTune, which lets companies tune applications for optimal performance before deploying them, was built with support for the Web-services protocols and open interfaces. Its offerings also support application servers from BEA Systems, IBM, and Oracle, all of which are expected to be used for building Web services. The vendor will continue to expand its products and business relationships related to Web services, says Jonathan Rende, VP of product marketing. It's early for Web services, but they're continuing to evolve, "and we have customers who want to test Web services on a project basis," Rende says. "It's absolutely critical that you have relationships with the thought leaders in this area, such as Sun, Microsoft, and Oracle."