Global CIO: Autonomy, Part II: Multivariant Testing And Killing The Hippos
In the second of a two-part series on Autonomy, find out how companies are putting its software to practical use.
Editor's Note: This is Part 2 of a two-part series. Part 1 can be found here.
"I always used to engineer our online-strategy meetings so that I'd be the most senior person in the room and could make all the decisions," said the very amusing John Peebles, VP of online marketing at Avis Budget Group, tongue firmly in cheek. "But now our new mantra is 'kill the hippos,' because we finally have the online tools that give us real, verifiable information on what works and doesn't work. So we don't need follow the hippo approach any more."
Now, before anyone rushes to call the ASPCA, John's not talking about going out and really killing any hippos; rather, he's talking about how rigorous multivariable testing allows online marketers to know -- not to guess or hope or expect, but to really know -- what approaches and which offers are grabbing customers' attention and dollars. And with such quantifiable information, companies no longer need to set strategy based on the Highest-Paid Person's Opinion -- they can stuff that particular hippo and lock it away in the trophy case with other outdated and ineffective online-marketing plans.
And the objective weapon that allowed Avis to slay that subjective hippo was Autonomy's Optimost multivariant testing (MVT) solution. And Peebles says he's actually quite happy that he no longer has to be so all-knowing all the time.
"It really has allowed us to become more customer-driven," Peebles said in a telephone interview. "We've begun taking feedback on our Budget [car and truck rentals] site, and are about to add it to our Avis site to give us very clear qualitative feedback. Every Monday morning, I come in and look through those user comments and then we take that plus the detailed data we have from our MVT work and we can create designs based by what really works.
"Before, we couldn't do that -- without an analytical tool, the best our people could do was tell you what they think will work."
In the past five years, Avis Budget has quadrupled its online revenue, which now represents more than 20% of the company's total revenue, Peebles said, and the intelligence Avis is gleaning from the Optimost tool offers the promise for additional growth going forward.
"Because we're getting more requests from customers to have GPS units in our vehicles, I had a meeting the other day with one of our finance guys to figure out what's the right price for the GPS-unit option," Peebles said. "And the finance guy said, 'We don't have much to go on from our early tests, but I don't want to lower the rate because we might end up lowering revenue and we don't know how well this'll catch on.'
"And I said, hey, we can test 10 different price points in each of 10 different locations and have real answers back for you in no time. And we're seeing possibilities like this for multivariant testing all over the company -- it's helping us break out of our corporate silos, just like that example with the finance guy.
"For example, we have 5 million unique visitors to our sites each month -- wouldn't that be a great source of market research?"
As noted near the end of last week's column, Peebles also attached a very specific revenue figure to his company's use of Optimost: "On the Budget.com site, we had a new design based on several iterations of multivariable testing. We tested over 20,000 different pages, colors, etc., and honed that number down to three completely different designs, and then we picked the best based on results that we knew were real, not guesswork. ... When we compared the revenue lift from that new design, it measured out to $5 million. These Autonomy tools and the multivariable capability give us the ability to measure the value of what we're creating. A few years ago, the only measurable detail we had to test for was price, but now we have millions of options and variables out there to test."
Peebles emphasized one other key benefit Avis Budget has realized in its use of Autonomy's Optimost solution: the tool's flexibility allows not only him and his team to take greater control over their sites and implement new data-driven insights more quickly, but also enables the company's IT team to focus more on strategic efforts and less on the "piddly" stuff that Peebles' marketing team can now handle.
"We relaunched the Avis.com site in January and it's just so far ahead of what it used to be," he said. "In the old version, Teamsite was there as the CMS but it was at that point really just an IT tool -- there was very little business involvement. For example, if we wanted to go in and make a very simple change on a rental form like changing 'First Name' to just 'First,' it would've taken six weeks to get that through the IT department. With the new system, it's gone from six weeks to six minutes. And I mean that.
"It's ideal for everybody," Peebles said, "because IT doesn't have to get involved in piddly things, and we don't get bogged down by the old way of doing things that just took too long."
Another big Autonomy customer that's had the same experience with a liberated IT department is Allstate Insurance, where senior Internet marketing strategist Lizzie Schreier said, "IT can now focus its time on higher-value projects, and an example of that is that I really need their help with integration. Right now, our sites are getting a lot more user-generated content and we've thrilled to have it, but I also need monitoring of that, and filtering, and I need to go out and find the right tools for those needs with IT as a partner and then integrate them."
Schreier said that as more and more of Allstate's customers engage with the company online, and do so in greater depth than ever before, new analytical-based processes such as adaptive targeting, measurement, and tracking are becoming "so much more critical to our success. So I need more-sophisticated tools for that, and I need to integrate them with our user-generated content rating tools."
The increasing popularity of users engaging with a site by adding their own content to it -- in discussion forums, blogs, rating forums, Q&A discussion groups, and more -- is one example Schreier offered of the overwhelming need for speed among businesses of all types.
"We're trying to change inside the company as fast as we can but it's also simply been a function of the market," she said. "Consumers are changing faster than we can change things, pushing from outside faster than we can push from the inside. So we're going to be very interested in tools that can help us make better decisions faster."
Noting that Allstate remains "a classic brick-and-mortar company" in spite of the huge gains its made in the online world, Schreier said, "Not just at our company but all over in the online space, it's getting harder and harder for companies to attract the right kind of consumers to their sites and, once they're there, to keep them interested, and a big part of that is that consumers are getting more and more savvy as days go on." And with the recession hammering at household budgets, consumers are more value-conscious then ever before, making the Web the perfect shopping partner for finding matches to product and pricing preferences.
"And again not just at Allstate but all over in this recession, as marketing budgets are cut, we're seeing more and more marketing moving from offline to online. And the automobile insurance market is an extremely competitive one, so we need to be able to deliver a great online experience that gets consumers to come in, engages them once they're there, and keeps them coming back."
Join us for a roundup of the top stories on InformationWeek.com for the week of December 14, 2014. Be here for the show and for the incredible Friday Afternoon Conversation that runs beside the program.