InformationWeek

Thomson & Thomson's Chase agrees. As a proprietor of legal research services for trademark, copyright, and script clearance, Thomson & Thomson developed its Saegis Web application to provide its subscribers with online access to research. Within the last 24 months, the Saegis user base has grown from 250 to 10,000.

"As growth continues to accelerate, we're focused on reconfiguring software and rearchitecting the Web infrastructure," says Chase. "As a result of a better-thought-out Web architecture and testing processes that included structured and repeatable functional regression and load tests, we're more confident in our system."

The "gut instinct" testing mentality that previously existed for determining readiness has been replaced with quantifiable facts about how the application will perform under real-world conditions. Chase reports that communication with the development team has improved as a result of leveraging automated tools. "Now when we uncover problems, we can back up our findings with the information our developers need to resolve the issues."

Many an IT manager has pitched a case for purchasing automated testing solutions and been challenged to prove the business value. The cost justification: How long before the investment outweighs the cost? After all, the foray into automating the testing process is expensive. Automated tools are costly, as are the qualified staff and hardware resources necessary to operate and maintain them.

According to Harper, Guild.com spent $100,000 on testing tools in 1999 and has allocated another $60,000 this year. This doesn't include the multiple quality-assurance hires, required hardware, or the outsourced testing consultant the company retains. This is why justifying the purchase and a commitment to automated testing have traditionally been uphill battles. However, the cost of not testing applications and eradicating errors carries an even higher price tag, a fact that's easing the budgeting pain.

OurBeginning.com CEO Michael Budowski has changed his views on the importance of automated testing. "When our IT team came to us with the suggestion of purchasing an automated testing solution via a hosted testing service, our executive team didn't give it much consideration because we were looking at it in terms of dollars and cents. However, when we stepped back and evaluated our seven-figure investment into the development of our site, and the fact that we were set to advertise during the Super Bowl, we decided to approve the purchase. "

In the end, it was a good business decision because the testing did find errors. After making the appropriate adjustments, OurBeginning.com was able to handle the 1,600% surge of visitors that flooded its site after the Super Bowl ad ran.

Budowski says that although getting buy-in prior to the Super Bowl was difficult, getting buy-in now wouldn't be nearly as tough. "I guess I had to have that experience to gain a first-hand technical perspective on the value that automated testing was able to deliver," he says.

What tends to most easily convince businesses of the need to establish formal automated test processes is a bad experience with an application providing poor service, or one that fails altogether.

The upside of that experience is that application quality becomes a bigger issue and gains the business support needed to survive. The downside is that adopting automated tools in response to a crisis can seem to worsen problems if it's assumed that having the tools will immediately equate to better application quality. Tools require process support and qualified people to run them.

According to Steve Marshall, CEO of T-Plan Ltd., a provider of test process-management tools, "typical client consulting engagements require 75% of the training to focus on the test process, with the remainder spent on learning how and where to leverage automated tools."

GlobalFulfillment.com's Wilkerson says automating testing "takes a lot of commitment, hard work, and adult supervision. You can't just install these tools and walk away. You must really understand what you're testing and be sure that processes are in place." Automating the testing process is a formidable challenge and it's an ongoing process, forging a continuous path of quality assessment and application improvement.

In response to growing business acceptance of automated testing solutions, vendors are strengthening their tool suites and emerging with flexible testing-service options. That's what Web services vendors such as Compuware, Cyrano, Mercury Interactive, Rational Software, and Segue Software are doing.

In addition to the traditional vendors responsible for turning automated testing into a distinct discipline, there exists a new breed of vendors that are providing automated testing solutions built exclusively for the Web. Those vendors include Keynote Systems, RadView Software, and Teradyne-owned RSW Software.

continued...page 3
return to page 1