Measuring The Value And Success Of SaaS 2

We look at real-world strategies to measure SaaS payback.

Michael Biddick, CEO, Fusion PPT

May 14, 2010

6 Min Read

InformationWeek Boardroom Journal - May 2010 InformationWeek Boardroom Journal Logo

Boardroom Journal Logo

Download the entire May 2010 issue of
InformationWeek Boardroom Journal,
distributed in an all-digital format (registration required).

Measuring The Value And Success Of SaaS

Measuring The Value And Success Of SaaS

Every CIO worth his or her salt knows that it's critical to gauge the potential impact that migrating to SaaS will have on the enterprise. But once SaaS is implemented, it's also important to measure whether it's delivering the value promised by providers, including reduced capital costs, faster application deployment, and reduced day-to-day IT support and operations tasks. CIOs also must determine if the SaaS application meets user and business needs in terms of features, capabilities, and performance.

This article will discuss ways that organizations can gauge where SaaS meets--and fails to meet--expectations and needs, including insights from our own experience as a SaaS consumer.

At our boutique IT firm, Fusion PPT, we recently decided to use SaaS for all of our business applications, from e-mail and collaboration to billing and application tracking. We use eight SaaS applications from seven vendors. Overall, we're pleased with them, but there are several areas we continually monitor.

Total cost of ownership is a key measure. SaaS providers tout the low acquisition costs of a service because customers don't have to make significant capital investments to get the app running, but SaaS costs will grow as you add users, buy into new features, and store more data with the provider. That's why you should revisit your original TCO analysis every 24 months.

When we calculate the TCO of SaaS, we evaluate both new and existing systems. For new systems, we compare the purchase price of the on-premises application to the SaaS application, costs of maintaining the application (including maintenance fees), as well as staff costs to support the application and infrastructure connected to it, such as backup and security systems. For existing applications, we also add the cost of migrating to the SaaS app and away from the existing one, especially extraction, validation, repair, and transformation of data costs.

In general, we've found that the TCO for on-premises utility applications (that is, applications that don't provide a compelling business differentiator, such as e-mail) are higher than the TCO for SaaS-based utility apps. In situations where we want to configure and customize an application, we've found more TCO parity between SaaS and traditional applications.

We also keep a close eye on support costs. While traditional software support isn't required with SaaS, we still have to add users, change passwords, create workflow, and perform other tasks. At Fusion PPT, these tasks fall to the business managers that use the SaaS apps. So far the workload is manageable, but we do monitor it so the business managers don't end up spending more time supporting SaaS systems than actually running the business.

We've also taken on a few unanticipated support costs. For instance, we decided to use a third-party SaaS backup provider for an additional layer of data protection on top of the redundancy we get from the primary SaaS vendors. We do this to ensure that if a SaaS provider suffers a massive catastrophe, our data will be protected. This choice has changed the original TCO models we developed, though only by less than 5% of the total support costs we anticipated across all of the SaaS providers.

Cost isn't the only measure we use to evaluate SaaS. Another important consideration is provisioning--that is, how quickly we can get the application into our users' hands. SaaS has a clear advantage over on-premises applications in this regard. Fact is, our users can start using a SaaS application within hours. By contrast, having to build out and deploy an on-premises application can take weeks or months.

Of course, it doesn't matter how fast you can get your users on board with an application if it lacks the features you need. When we were evaluating billing applications, we ran a conventional analysis to determine if the SaaS providers on our list offered the right set of capabilities. We chose a provider that met our needs at the time, but our analysis didn't anticipate a billing requirement we ended up needing to meet a demand of one of our own customers. The result: We had to migrate from one SaaS provider to another. We spent a substantial amount of time configuring and setting up the new SaaS system--more than 200 hours. This is 100 times the investment we made with the first billing SaaS provider, to meet the new requirements.

Just because an application is delivered via a provider doesn't mean you won't have to spend time on customization and configuration. In our experience, SaaS applications such as e-mail, collaboration, and CRM have required only a few hours of configuration, but the SaaS billing application required a more substantial investment on our part to fully meet our business needs.

This leads to another post-deployment lesson learned: the importance of data portability. If you're dissatisfied with the SaaS provider's application or service, you need to be able to remove your data from the application. In the case of our billing app, the data was stored in a format that made it easy to port it from one SaaS billing provider to the other.

However, we've seen examples from our consulting engagements where data hasn't been portable, effectively locking in a customer to a SaaS provider. In addition, if you have to move from one SaaS provider to another, the customizations you made will be lost.

Another area we continually evaluate is data integration with disparate SaaS providers. While data integration in a non-SaaS environment also can be challenging, particularly around identity management and business process management, it's just as difficult with SaaS (and impossible with some providers). Currently, we're evaluating a cloud-based data broker that could allow us to exchange data among the SaaS providers we use. While this integration isn't vital to our business, it would help automate and reduce manual processes. If your business requires tight systems integration, you will need to invest in platforms to exchange data in a SaaS environment.

To read the rest of the article,
Download the May 2010 issue of InformationWeek Boardroom Journal

About the Author(s)

Michael Biddick

CEO, Fusion PPT

As CEO of Fusion PPT, Michael Biddick is responsible for overall quality and innovation. Over the past 15 years, Michael has worked with hundreds of government and international commercial organizations, leveraging his unique blend of deep technology experience coupled with business and information management acumen to help clients reduce costs, increase transparency and speed efficient decision making while maintaining quality. Prior to joining Fusion PPT, Michael spent 10 years with a boutique-consulting firm and Booz Allen Hamilton, developing enterprise management solutions. He previously served on the academic staff of the University of Wisconsin Law School as the Director of Information Technology. Michael earned a Master's of Science from Johns Hopkins University and a dual Bachelor's degree in Political Science and History from the University of Wisconsin-Madison. Michael is also a contributing editor at InformationWeek Magazine and Network Computing Magazine and has published over 50 recent articles on Cloud Computing, Federal CIO Strategy, PMOs and Application Performance Optimization. He holds multiple vendor technical certifications and is a certified ITIL v3 Expert.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights