Catalina Marketing Aims For The Cutting Edge Of 'Big Data'
No. 5 company in our 2011 InformationWeek 500 ranking is fast, too, delivering insights within a second of a transaction.
At one point, Catalina's CRT team had doubts that the project would fly because they were once again pushing the limits of available technology. To speed development, Catalina was trying to adapt a credit-authorization server product normally used by banks and oil companies with many locations. Catalina didn't need to do credit authorizations, but it wanted to take advantage of the store-to-headquarters transaction-posting functionality. That software normally runs on big iron servers, but Catalina wanted to run higher-than-normal transaction volumes on a cheaper class of commodity blade servers. Early performance specs fell far short of expectations.
Initial tests were running at fewer than 100 transactions per second, where they needed to exceed 300. "We knew we didn't have a business model unless we could get it to run faster on a Linux-based platform using low-cost blade servers," Williams says.
Catalina's developers cracked open the software and stripped out every scrap of unnecessary code tied to credit authorizations. That worked, and the team eventually pushed system performance up to about 450 transactions per second. It was a benchmark required to support retailers with thousands of locations and dozens of checkout lanes per store.
By the end of this year, Catalina expects to deploy CRT across 22 of its top retail customer chains, accounting for about 85% of Catalina's retail transactions.
Driving The Business
Big data still needs to be fast
Technology is obviously hugely important to Catalina, and the employee numbers bear that out: Nearly half of the 500 people at headquarters work in IT. "We are a technology company, and IT is responsible for the innovation of the business," says Williams, who looks to hire experienced, versatile team players. "I'm not looking for people who have worked in large development shops and are used to working in silos of development," he says. "Our teams are cross-functional, and they have to handle whatever needs to be done."
Catalina typically assigns what Williams describes as "agile" teams of 12 to 15 people to projects such as the in-database analytics and CRT ones. Goals change fast. "I'm amazed at how many people in technology like to have clearly defined requirements," he says. "We're a sales company, so I guarantee you the targets are going to move, and you need to plan for them to move."
Catalina IT is akin to a product development team, as Williams describes it, working with brand development people to explore what's possible. "My team just loves that," he says, "because they can take on these projects and then go see them really work out there as a consumer."
6 Tools to Protect Big DataMost IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Big Data Brings Big Security ProblemsWhy should big data be more difficult to secure? In a word, variety. But the business won’t wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more.