Put Your Test Lab In The Cloud

Putting software through the paces of a cloud-based development, testing, and quality assurance lab is one of the best ways to try out a cloud environment without having to worry about the data control and security issues of production environments.

Public cloud environments like those from Amazon, Microsoft, Joyent, and soon Hewlett-Packard let developers rapidly scale lab infrastructure and deploy the needed tools without making big up-front capital investments. But if they're not carefully managed, public cloud lab costs can escalate, ratcheting the total cost of ownership higher than for on-premises or private cloud labs.

We found this out with a lab we ran in the Amazon Web Services cloud. At the beginning of the month, one of our developers stood up a new application that used a lot of CPU and storage, and he accidently left it on. Lab costs that month were up tenfold. Amazon kindly sent a note thanking us for relying on AWS for more of our needs, but more proactive management was what we really needed.

But private cloud labs are complicated and can be costly to set up. Ultimately, your organization's security requirements, budget constraints, and ability to deal with complexity will determine which way you go.

Why A Cloud Lab

From a business perspective, organizations can set up public or private cloud labs faster than on-premises labs and shift lab costs from capital to operational expenses. By putting your lab in the cloud, you give developers a self-service portal where they can create, replicate, change, and delete entire software development projects and test stacks on demand. Developers and testing engineers can use various databases, operating systems, browsers, application builds, and middleware combinations for the configurations they need--all without involving the IT folks. They also can create new release stacks on demand and ensure that they're consistent through release cycles.

With a cloud lab, developers also can quickly re-create a complex bug or production problem, take a snapshot, and then make changes, run tests, and compare the results to the original snapshot.

A cloud lab supports mobile workers more effectively than a conventional one, because development tools can be accessed from anywhere with a network connection. It also makes it easier to have a different mix of team members on various projects. You can easily implement customized access polices for each role on a team and give outside contractors restricted roles and rights compared. Team members can share configurations, templates, assets, and files specific to their projects, and collaborate during the entire development cycle.

Although the cloud model brings users together quickly, fosters collaboration, and maintains consistency while increasing productivity, it's still costly and time consuming to build all the software services you'll need. It's often better to use commercial off-the-shelf software in a private cloud or software-as-a-service tools in a public cloud. There are several vendors that offer cloud lab products, including CollabNet, Electric Cloud, eXo, and Skytap Cloud.

Strategy: Development Labs In The Cloud

Our full report on putting a lab in the cloud is available free with registration.

In this report, you'll get

Get This And All Our Reports

Go Public

But if you want to get started quickly, the public cloud is your best option. Scale and automation are what have made public cloud service providers successful. These providers tap into a massive user community, unmatched in even the largest enterprise environments. They've invested millions of dollars automating data center processes--everything from service requests to provisioning to alert detection. It's relatively easy to take advantage of public cloud infrastructure services and host your development and testing tools inside these environments.

While the public cloud offers fast startup with no capital investment, the TCO over time may be higher compared with on-premises and private cloud lab environments. Fees from your cloud lab service provider can be difficult to monitor and control, and, as mentioned earlier, end-of-the-month surprises are a constant concern.

Software tools can help companies manage public cloud lab costs, but they can be expensive and don't make sense unless you're spending several thousands of dollars a month in lab fees. Uptime Software's UptimeCloud, for example, is an online service that monitors Amazon virtual server use, calculates existing charges, and projects them well before the end of the month. You can build in limits so that going over budget triggers an alert.

Portability also is a huge concern in the public cloud, particularly for a development lab. Vendors have a certain degree of lock-in with their SaaS applications in the public cloud lab, making it difficult to transfer workflows if you have to change providers. If you're tracking bugs and other issues, you may not be able to easily export and import that tracking into another system.

In addition, some public cloud providers charge for data coming into and out of their cloud offerings, and those costs can add up. When planning to move your lab into a public cloud environment, consider the total amount of data you'll be moving in and, more important, how much data you'll have to deal with if you ever want to get it out of the cloud relationship.If you're putting a lot of data into a cloud lab or are using valuable code there, you'll need a portability strategy.

Security is another big concern. Without formal processes to deal with data security, applications may be abandoned on virtual machines and live perpetually in a public cloud. This approach isn't wise, especially if you're dealing with sensitive, business-critical applications or data. You'll need to inventory the applications and data that must reside in a public cloud and ensure that your security team is OK with the approach.

Key Benefits

Of A Cloud Lab

Provides development and test templates compliant with IT policy Centrally creates users, roles, access control lists, and permissions Tracks usage by month, user, and project; can implement chargebacks if needed Shifts lab from capital to operating expense Avoids technology refresh expenses altogether (public cloud only) Establishes hybrid cloud architecture to connect to in-house data centers Assigns individual-, project-, and group-level quotas for machines, storage, and networks Audits and ensures that compliance policies are followed

Stay Private

Cost control is easier in a private cloud lab. Your data and apps will be more secure there as well. A private cloud also centralizes shared resources for large environments. But the only way to achieve cost savings in a private cloud lab is to have large-scale adoption. You'll have to get most of your organization using your private cloud lab, and for that to happen, you'll need it to be automated, which is the key to any successful cloud lab deployment, especially if cost reduction is a priority.

Besides automation, you'll need other key technology elements, such as bug tracking, collaboration, and asset management (see chart below). As with any private cloud environment, you'll have to decide how far you want to take your lab. Are you just after platform service and infrastructure service capabilities, or do you want to offer software configuration management (SCM) and bug tracking as hosted solutions? How much control do you want users to have over provisioning resources? Once they're provisioned, how much management and monitoring of these resources will be required? Do you need chargeback or usage-based reporting? How quickly do you need to tear down and rebuild environments? Can you replicate the amount of data and traffic needed in the lab?

A private cloud lab may be costly and complex, but it has advantages over public cloud options. It gives you the flexibility to replicate your production environment, which may be necessary to test some applications.

Unfortunately, many tools designed for software development and testing, including SCM tools, aren't readily portable to a private cloud. This can be a problem, as having a full suite of development applications and the control mechanisms to manage them is key to getting value from a private cloud lab.

If you find you must mirror your production environment in the cloud, you'll need a multitier application with clustering and failover support--and that's expensive in the public cloud, especially for large environments. You might need multiple networks in various configurations, and you might need to add multiple network adapters to each virtual machine, connect them to different networks, configure routing policies, and ensure that you're getting an accurate depiction of the production environment. One tool that can help is centralized configuration management automation software, which works across different applications, networks, and servers.

Setting up a lab in a private cloud also lets you control security, as well as the code and other data in the lab, making it the best option for sensitive data and applications. Private clouds also keep you in control of high-risk functions such as disaster recovery and operational continuity.

The cost to scale also will dictate whether you locate your lab in the public or private cloud. But there's no fast and easy formula for figuring that out. The decision depends on your specific needs. Online gaming software developer Zynga found it could do the same amount of work in its private cloud as it had been doing on AWS but with one-third the number of servers. AWS's general-purpose servers are designed to run a wide variety of workloads and don't do one job supremely well. Zynga optimized its servers for specific roles--database access, Web server, and game logic execution--and ended up using fewer of them. Like Zynga, you'll want to do a TCO analysis, comparing public cloud services with a build-it-yourself private cloud.

First Steps

If you're ready to move your lab to the cloud, start by transitioning some applications or services there. Public cloud lab services are easy and cheap to try out. You can customize the networking, hardware, and storage properties of the public cloud lab to match your in-house test labs, and you can easily scale these configurations to hundreds of concurrent machines that represent thousands of users. Each configuration is completely isolated, so you can run multiple environments concurrently without impacting other users.

Since most software applications are deployed in a multitier model--database, application, and Web tiers--reproducing complex bugs and resolving them quickly is a huge challenge in any cloud lab, public or private. Development and test teams often establish multiple test labs with identical environments to do this, but multiple test labs are costly to set up, underutilized, and time consuming to manage.

One way to make this task easier is to use standardized development and test environment templates like the one provided by Skytap Cloud, which lets developers quickly provision new environments. With templates, developers can perform destructive tests, whereby they change database settings, application settings, and router policies; see how the application behaves; and then return to the baseline setup by redeploying the source template. Because each configuration is encapsulated, you can install tools, test application behavior, and restore the clean version. An entire virtual data center with all of its machines, data, memory state, network settings, and application state can be retained within a template. As you move through the development life cycle, you can preserve points where you encounter problems. Templates can be shared with remote teams for rapid bug reproduction and resolution.

Three Areas To Assess

While it's exciting to jump right into a cloud lab, pay attention to three underlying elements:

>> Standardize. The more standardized your existing lab is, the easier it will be to move it into the cloud. If your lab is highly customized and uses a number of different platforms and application stacks, moving it into a cloud may not get you much. Standardization enables widespread adoption of cloud resources within your company, without which you'll be unlikely to see cost savings.

When you launch new software development projects, take a firm stand on standardization. If you can't standardize across the enterprise on development and test systems, you won't have enough scale to capitalize on the cost savings a private cloud offers, and it will take a lot of effort to set up all the distinct environments you need in the public cloud.

>> Get the process right. It's easy to stand up resources in a virtual environment, but who turns them off? The easier it is for users to request services from the cloud, the more complex the back-end process needs to be to monitor and decommission them when they're no longer in use. This is especially true if the meter is running in a public cloud.

From the service request to provisioning and managing the lab, you must have established processes if you're going to automate the tasks involved and achieve the benefits of rapid, on-demand provisioning and scale. Problem- and error-control processes that detect, log, categorize, investigate, document, and repair problems as they occur are particularly important. Having an integrated change management process, including a change advisory board that will keep stakeholders aware of lab changes and extend production changes to the lab, also is vital.

The best server provisioning, orchestration, and management tools won't compensate for a lack of process. When you shift those tools to the cloud, they won't get you the cost savings you're looking for and may mask some of the problems the lack of a good process is causing.

>> Establish governance. In both a public and a private cloud lab, you must have asset management, provisioning, and configuration management tools to monitor and track virtual resources and report on key indicators. The ability to manage lab capacity is just as important in the cloud as in conventional on-premises labs, but you'll need different tools and processes. Unfortunately, available management tools are aimed at virtualized production data centers and aren't that useful for a cloud lab. They often lack the automation controls that will help companies realize the cost savings they're seeking in the cloud lab. Vendors like VMware, Hewlett-Packard, and IBM have or plan to have tools that can help; custom tools are also an option.

You'll also need a versioning control tool to keep everyone on the development team focused on the same baseline version of the product they're working on and ensure that edits are being done on the correct build. Concurrent Versions System provides this functionality, and many other tools can be tailored. IBM, Spectrum, and other vendors have SCM tools that provide version control and other support in a virtualized lab environment.

The bottom line when you deploy your cloud lab is to continually monitor costs and keep re-evaluating your TCO model. Public cloud labs can be lifesavers for companies that need to set up and scale fast. For others, security, control, portability, and integration issues will push them to private cloud labs. Know the business case before you make a move and continue to evaluate whether the cloud lab you choose is living up to expectations.

Continue to the sidebar:
HP Takes Developer Slant With Cloud Offering

InformationWeek: Apr. 9, 2012 Issue

Download a free PDF of InformationWeek magazine
(registration required)