How To: APM In SaaS, Cloud 2

Just because you're using software as a service or hosting systems with a public cloud provider doesn't mean you're off the hook for performance problems. On the contrary: A critical application is a critical application, whether it lives on your own servers or in a mega data center in Singapore. It's a conundrum your security group is likely also coming to grips with: How do you guarantee highly functional, regulatory compliant computing when the infrastructure is entirely out of your control?

The security guys are on their own. For application performance management (APM) suite jockeys, what will change as applications that impact the bottom line move into the cloud is the set of metrics to be monitored. You'll need to transition from twisting dials to working closely with your cloud provider to aggregate data and give the business a real-time view of what matters. You'll also need the grit to abandon APM systems that just aren't working. Throwing good money after bad will only reinforce business leaders' tendency to see internal IT as hidebound, behind the times, and replaceable.

We've tracked APM usage levels for years. In our full InformationWeek Analytics APM report, we compare results from our August survey of 379 business technology professionals with what 320 respondents told us in April 2009. The top overall trend: consistency. The number of survey respondents who say their companies have no plans to use APM tools or systems held steady at 21%. Also fairly consistent from survey to survey were the frequency of performance problems, perceived criticality of application services, and user/customer tolerance (or lack thereof) to application service outages. The biggest change: the top reason companies don't plan to use APM. In 2009, 36% of survey respondents cited the cost, and the same percentage noted their lack of expertise. But in 2010, the percentage saying they don't have the needed expertise jumped a full 14 points, to 50%. Cost is up five points.

We think the complexity involved in monitoring applications hosted offsite and in private clouds is contributing to the shortage of expertise. The huge field of vendors pitching APM products of varied architectures is also a source of anxiety.

Given the confusing and shifting product landscape, architectural complexities, and manpower issues, is APM worth the effort for companies moving aggressively toward the cloud? For both cloud and in-house environments, an end-to-end APM initiative will likely make sense only for enterprises that have more than $50 million in revenue or that run 10 or more mission-critical applications that directly affect the bottom line. In a cloud environment, APM is trickier--simpler with SaaS providers, but harder for infrastructure-as-a-service and platform-as-a-service private and public clouds. Still, you need to ensure that the cloud vendor isn't costing you business and that APM is part of a larger service management strategy.

Unfortunately, too many companies find APM benefits out of reach because they haven't figured out how to use monitoring to prove in-house applications are meeting business objectives. If you can't execute critical functions like configuration management, performance monitoring, and capacity planning when you control the network, doing so for applications hosted in the cloud will be a mighty tall order.

Conventional APM is already tough. In a public cloud environment, you lose control of, and visibility into, the underlying infrastructure--and the integrated monitoring that comes with that control. For now, unless you're Google or Walmart, forget placing APM hardware or software in a SaaS or public cloud network. PaaS and IaaS providers even typically limit your ability to monitor apps on their platforms and infrastructures. Thus, problems can be detected only with synthetic transaction software simulating a user accessing the application. When you find issues, you may as well throw a dart to figure out the root cause: servers, network, and storage bottlenecks, or the Internet between your users and the public cloud.

Different Mind-Set

But here's the thing: That lack of insight may end up being a blessing in disguise. You moved to the cloud to lower costs, limit complexity, and increase the scalability of systems. Enterprise-class APM tools can be incredibly costly, and getting the most out of them is a labor-intensive process. APM in the cloud will require a different, dare we say Zen-like, mind-set. Instead of hundreds or thousands of IT teams attempting to monitor "their" environments inside the cloud, successful cloud providers will step up and provide their customers with much more transparent visibility into component health. Enterprise IT teams can then transition from tactical monitoring to strategic data aggregation.

Of course, that means placing a lot of faith in the provider. How well SaaS and cloud vendors do here will be pivotal as they ask enterprises to trust them with crucial systems. We predict that the ability to analyze and collect data from multiple streams and provide a holistic service view will be a major differentiator. Fortunately, cloud providers can take advantage of the scale inherent in their environments to justify the large costs involved in the purchase, deployment, customization, and management of APM systems powerful and granular enough to monitor thousands of customers' applications.

For CIOs, instead of having teams scrambling to deploy agent technology to track discrete values of components inside the cloud, they'll transition to receiving data sets from internal and external providers. The work of the IT organization will be to aggregate this data and filter it through the proper metrics to provide an end-to-end service-health view that's truly relevant to the business.

That's our recommendation, anyway.

However, in our survey, 60% of respondents say they're extending their APM tools to monitor SaaS or public cloud applications. Given the limited visibility that APM tools provide in this situation, we were surprised by this finding. Trying to use standard enterprise APM suites in the public cloud won't get you very far, since you can't sniff out the root causes of problems. Moreover, as distributed workloads move into cloud environments, the fundamental application architecture is poised to change dramatically, to systems that operate divorced from physical hardware. Clearly, our data collection techniques must shift as well.

So do you need to dump existing APM tools when critical application functions move into the cloud? Perhaps, and we certainly don't recommend spending money to launch a new in-house APM initiative if your application road map is heavy on SaaS/cloud. Instead, anticipate the new model of providing an end-to-end service health view by determining the right APM metrics.

Whether your applications are in the cloud or snug in your own data center, APM is all about the ability to quickly determine what's going wrong, where the problems are, who is impacted, and how the company's bottom line is affected. That means knowing the relationships among all IT components--physical, cloud, and SaaS--and how they affect business services. Near term, the ability to obtain key performance indicators and service-level agreement data for all outsourced applications is critical.

The ultimate goal of APM in the cloud era is a dashboard that presents a holistic vision of the entire IT environment in real time and from a historical perspective. This may seem like a pipe dream, but keep your eyes on that goal when evaluating APM, SaaS, and cloud vendors.

But What Do We Do Now?

OK, so you're sitting on a few hundred thousand dollars in legacy APM framework or point management tools, maybe even a manager of managers or business service or IT service management suite. Do you try to wring out value as you migrate the enterprise application architecture to virtualization and SaaS/cloud delivery models?

Frankly, as painful as it may be, stepping back and re-evaluating is a better choice than stubbornly piling on staff hours and licensing fees. Politically and economically, that's a hard stance to take. We often see cases where an APM deployment isn't living up to expectations, but everyone is afraid to kill the project They see it as safer to continue to scratch and claw for incremental progress toward a goal that grows ever more elusive as the company adopts more SaaS. And in a big-picture sense, this unwillingness to kill projects that need killing is a reason some business leaders are doing end runs around IT.

Long term, the data-collection fundamentals of APM will remain the same: Agents installed on application tiers, the operating system, virtualized servers, database components, and even client workstations will detect problems affecting the application, such as memory usage, CPU, and network activity. You may be tempted to forgo the client agent that tracks performance glitches resulting from end user workstations, but where customer satisfaction counts, this is critical. Synthetic transaction monitoring detects performance snags during off-peak hours and finds problems users may experience but not report, while network probes capture actual user data so you can monitor and baseline the customer experience and detect problems during an application slowdown.

An Ounce Of Prevention

Cloud computing, frankly, has turned the APM market on its ear. Our take is that most enterprises will eventually have hybrid public/private clouds, combining internal and external resources to host critical applications. Until the dust settles, instead of striving to monitor and measure every last aspect of every application, turn your attention to how such an architecture can help head off performance problems: Cloud bursting may allow the rapid expansion of resources to meet demand spikes, preventing slowdowns before they arise. WAN optimization can allow better access to cloud applications via caching and compression over the network. These are just a few ways the cloud can deliver better application performance. But to take advantage, we need to start changing our psychology; vendors that offer nimble systems that are quick to deploy and easy to manage are the future of APM.

Some innovators are starting to respond. APM vendor SL Corp. provides metrics to evaluate the critical infrastructure components required to run a cloud environment. One example is tough-to-monitor grid technologies like Tibco ActiveSpaces, Oracle Coherence, and IBM WebSphere eXtreme Scale. SL's RTView offers cloud providers the ability to expose data to enterprise customers, while SL Historian can provide exposure at different levels of data granularity to different customers. The products' ability to cache and provide fast access to data for presentation as well as root cause analysis is suited to the cloud. Compuware's Vantage also is focused on correlating data sources across a variety of platforms and looking at real-time and historical data.

To select the right APM system, ask yourself six questions:

>> Are you heading into the cloud? Public cloud platform, infrastructure, and software providers will eventually offer better APM metrics, but agent deployment always will be tricky. APM systems that depend heavily on agents won't extend well into the cloud.

>> How deep will the APM tool go? Today, the No. 1 differentiator is whether the system has the ability to automate some resolution tasks or integrate with other automation tools to provide that functionality.

>> Does it play well with others? Underlying infrastructure--storage, network, OSs--can cause application problems. While the APM system may not monitor these elements, it should provide a data feed to another tool, or pull in external information to get a complete picture of the environment.

>> How big is the footprint? Are agents needed? Is the APM system delivered as an appliance? The terms "passive" and "active" are still thrown around a lot. Make sure you understand the system's reach.

>> Will the APM tool show the numbers? Especially in the cloud, a lot of information will come from baselining, yet companies still struggle to understand what constitutes a good or bad performance metric. Evaluate out-of-the-box reports, and ensure they can compare baselines to actuals and show trending.

>> Will tuning be a full-time job? Some APM tools are very complex, and extending them into the cloud isn't likely to help. An elegant system that provides core functionality will be much more beneficial than one with dozens of bells and whistles that's not deployed fully or that you can't update without expensive help.

Michael Biddick is president and CTO of Fusion PPT. His focus areas include ITIL v3, APM, cloud computing, and program management. Write to us at [email protected].