The Cloud's Next Big Thing: Software Testing

IBM and Hewlett-Packard have jumped into the market for software testing in the cloud, a space where a handful of startups already offer alternatives to on-premises testing. It's an area of cloud computing where the barriers to entry are relatively low for developers and IT departments.IBM estimates that one third to one half of computer hardware in some large companies is used for software development, testing, and staging. That's a lot of computer processing power and expense devoted to what's an important but non-business critical requirement. It's the type of work, given data center economics, that might be better suited to cloud services.

"The non-production infrastructure is usually as large as the production infrastructure," says IBM's cloud computing CTO Kristof Kloeckner. "These test environments are grotesquely underutilized and, when needed, you can't get it."

Cloud computing is well suited for software development and testing for several reasons. The model supports group collaboration; new servers can be launched and decommissioned as needed; software development projects tend to be temporary in nature, with start and end dates; and some of the governance issues don't apply because sensitive corporate data isn't involved. Also, cloud testing enables new approaches and capabilities, such as crowd-sourced testing.

I've talked to a bunch of cloud testing companies over the past few months. Here are some of the options:

IBM: Company's Smart Business Development & Test service, offered from the "IBM Cloud," is aimed at developers who use IBM's Rational toolset. IBM's CloudBurst for Development & Test appliance (price $207,000) can be used for on-premises testing.

SauceLabs: This Silicon Valley startup lets developers test Web apps using the Selenium test suites, with an emphasis on speeding up the process. Selenium is a platform for testing Web apps written in a variety of programming languages. SauceLabs, which is hosted on Amazon Web Services, lets you record video of browser action to troubleshoot and document tests.

Skytap: Promotes a hybrid cloud model where customers use its testing services in combination with their own on-premises tools and processes. Features include a library of operating systems, databases, and other software for creating drag-and-drop test environments. Skytap last week partnered with Hewlett-Packard to offer software testing through HP's Quality Center. uTest: Uses a community of professional testers, i.e. crowd sourcing, to help developers identify software bugs and other issues. Types of testing include functional testing, load and performance, and usability. Customers can view the backgrounds and profiles of 18,000 on-demand testers and grade the quality of their work.

VMLogix: Specialist in virtual machine management announced this week VMLogix LabManager, Cloud Edition, which runs on Amazon's EC2. Can be used to create virtual labs for testing software stacks in the form of Amazon Machine Images, with a central repository, team capabilities, and audit trails. Works with VMLogix's on-premises platform to create hybrid software lab environments. Service in beta now; GA in September.

Zephyr: Provides "testing lifecycle management." Zephyr 2.5 platform provides testing desktops and dashboards served from a hosted Zephyr Management Platform, which incorporates testing applications and APIs. Capabilities include test case creation, test automation, defect tracking, collaboration, and project management.

That's a quick overview of a half-dozen software-testing-as-a-service providers; I'm sure there are others and, given the trend, that there will be more. How well do these services work? Feel free to share your experiences here.