Measuring The Value And Success Of SaaS - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Cloud // Software as a Service
News
5/13/2010
02:58 PM
Connect Directly
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Measuring The Value And Success Of SaaS

We look at real-world strategies to measure SaaS payback.

Every CIO worth his or her salt knows that it's critical to gauge the potential impact that migrating to SaaS will have on the enterprise. But once SaaS is implemented, it's also important to measure whether it's delivering the value promised by providers, including reduced capital costs, faster application deployment, and reduced day-to-day IT support and operations tasks. CIOs also must determine if the SaaS application meets user and business needs in terms of features, capabilities, and performance.

This article will discuss ways that organizations can gauge where SaaS meets--and fails to meet--expectations and needs, including insights from our own experience as a SaaS consumer.

At our boutique IT firm, Fusion PPT, we recently decided to use SaaS for all of our business applications, from e-mail and collaboration to billing and application tracking. We use eight SaaS applications from seven vendors. Overall, we're pleased with them, but there are several areas we continually monitor.

Total cost of ownership is a key measure. SaaS providers tout the low acquisition costs of a service because customers don't have to make significant capital investments to get the app running, but SaaS costs will grow as you add users, buy into new features, and store more data with the provider. That's why you should revisit your original TCO analysis every 24 months.

When we calculate the TCO of SaaS, we evaluate both new and existing systems. For new systems, we compare the purchase price of the on-premises application to the SaaS application, costs of maintaining the application (including maintenance fees), as well as staff costs to support the application and infrastructure connected to it, such as backup and security systems. For existing applications, we also add the cost of migrating to the SaaS app and away from the existing one, especially extraction, validation, repair, and transformation of data costs.

In general, we've found that the TCO for on-premises utility applications (that is, applications that don't provide a compelling business differentiator, such as e-mail) are higher than the TCO for SaaS-based utility apps. In situations where we want to configure and customize an application, we've found more TCO parity between SaaS and traditional applications.

We also keep a close eye on support costs. While traditional software support isn't required with SaaS, we still have to add users, change passwords, create workflow, and perform other tasks. At Fusion PPT, these tasks fall to the business managers that use the SaaS apps. So far the workload is manageable, but we do monitor it so the business managers don't end up spending more time supporting SaaS systems than actually running the business.

We've also taken on a few unanticipated support costs. For instance, we decided to use a third-party SaaS backup provider for an additional layer of data protection on top of the redundancy we get from the primary SaaS vendors. We do this to ensure that if a SaaS provider suffers a massive catastrophe, our data will be protected. This choice has changed the original TCO models we developed, though only by less than 5% of the total support costs we anticipated across all of the SaaS providers.

Cost isn't the only measure we use to evaluate SaaS. Another important consideration is provisioning--that is, how quickly we can get the application into our users' hands. SaaS has a clear advantage over on-premises applications in this regard. Fact is, our users can start using a SaaS application within hours. By contrast, having to build out and deploy an on-premises application can take weeks or months.

Of course, it doesn't matter how fast you can get your users on board with an application if it lacks the features you need. When we were evaluating billing applications, we ran a conventional analysis to determine if the SaaS providers on our list offered the right set of capabilities. We chose a provider that met our needs at the time, but our analysis didn't anticipate a billing requirement we ended up needing to meet a demand of one of our own customers. The result: We had to migrate from one SaaS provider to another. We spent a substantial amount of time configuring and setting up the new SaaS system--more than 200 hours. This is 100 times the investment we made with the first billing SaaS provider, to meet the new requirements.

Just because an application is delivered via a provider doesn't mean you won't have to spend time on customization and configuration. In our experience, SaaS applications such as e-mail, collaboration, and CRM have required only a few hours of configuration, but the SaaS billing application required a more substantial investment on our part to fully meet our business needs.

This leads to another post-deployment lesson learned: the importance of data portability. If you're dissatisfied with the SaaS provider's application or service, you need to be able to remove your data from the application. In the case of our billing app, the data was stored in a format that made it easy to port it from one SaaS billing provider to the other.

However, we've seen examples from our consulting engagements where data hasn't been portable, effectively locking in a customer to a SaaS provider. In addition, if you have to move from one SaaS provider to another, the customizations you made will be lost.

Another area we continually evaluate is data integration with disparate SaaS providers. While data integration in a non-SaaS environment also can be challenging, particularly around identity management and business process management, it's just as difficult with SaaS (and impossible with some providers). Currently, we're evaluating a cloud-based data broker that could allow us to exchange data among the SaaS providers we use. While this integration isn't vital to our business, it would help automate and reduce manual processes. If your business requires tight systems integration, you will need to invest in platforms to exchange data in a SaaS environment.

Maintaining Satisfaction Levels

Applications that are low cost and easy to deploy won't be any good if they don't keep users happy. We've had minor complaints with our SaaS providers, like our hosted SharePoint sites not notifying us when documents were updated, or our SaaS-based applicant tracking system not sending out automated e-mails to candidates. For each problem, we found the vendors extremely responsive and able to resolve the issue within the day--better than at many IT organizations. We also conducted no training on any of the systems, and surprisingly, we haven't had more than a handful of questions, all resolved within the team because someone else figured out how to do it. People love the ability to access SaaS apps from anywhere, and the intuitive feel of the interfaces.

Success with SaaS is also measured by events that don't occur, such as security breaches, major outages, and performance hits. For high-value systems like CRM and accounting, we seek vendors that comply with the SAS 70 Type II standard, and we review the vendors' security documents. A SaaS vendor's data retention and restoration practices must align with our own. Problems like system failure may be straightforward, but what about accidental deletion within the SaaS environment? How long are the backups to be retained by the SaaS vendor? If there's a disaster at the SaaS provider's data center, how long will it take the provider to restore data for customers? If restoration of data is requested by a SaaS user, how long will it take?

How much extra bandwidth you'll need to cover the additional throughput required for SaaS depends on the types of applications, but network failover and connectivity are critical. Be sure you've got sufficient network capacity to support your SaaS environment.

In general, we're pleased with our SaaS providers; so far, they've been responsive to our issues. But as the SaaS industry grows and providers need to support more end users, service quality could become diluted, even as customer demands for better, more customized services increase.

If our own satisfaction levels erode, we don't want to be held hostage. For this reason, it's critical to constantly evaluate your SaaS providers and hold their feet to the fire.

Michael Biddick is chief technology officer of Fusion PPT, a technology consulting firm, and a contributing analyst to InformationWeek Analytics.

You can write to us at [email protected].

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

News
Remote Work Tops SF, NYC for Most High-Paying Job Openings
Jessica Davis, Senior Editor, Enterprise Apps,  7/20/2021
Slideshows
Blockchain Gets Real Across Industries
Lisa Morgan, Freelance Writer,  7/22/2021
Commentary
Seeking a Competitive Edge vs. Chasing Savings in the Cloud
Joao-Pierre S. Ruth, Senior Writer,  7/19/2021
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Monitoring Critical Cloud Workloads Report
In this report, our experts will discuss how to advance your ability to monitor critical workloads as they move about the various cloud platforms in your company.
Slideshows
Flash Poll