2 Ways Big-Data Analysis Pays Off
Interclick's experience shows that getting answers within seconds can improve productivity, but sometimes it's also the only way to reach best prospects before they buy.
No doubt you've read about the advantages of modern, big-data analysis platforms. Queries that once took days or hours can be cut down to minutes or seconds. So what's the big hurry? Ad services company Interclick’s use of the platform illustrates the two key ways companies benefit from fast, big-data analysis.
Lots of people talk about the need for speed, but you don't often hear about scenarios in which insights are acted upon within minutes of their discovery. Let's call that a near-real-time scenario. Much more common are examples in which fast querying frees up time for more analysis -- either exploring more data or doing deeper queries. Let's call that a productivity scenario.
More Software Insights
- The Critical Importance of High Performance Data Integration for Big Data Analytics
- Mobile DevOps: Achieving continuous delivery with multiple front ends and complex backends in Banking, Financial Services, and Insurance
- Managing Volatility through Smart Inventory Planning
- Agile Maturity Report: Benchmarks and guidelines to improve your effectiveness
For Interclick, a fast-growing New York-based firm that strives to put the right online ads in front of the right people at the right time, both scenarios come into play. The company launched its core Open Segment Manager (OSM) platform in 2009. OSM is essentially a data-driven service that helps advertisers reach targeted segments of would-be customers. The database combines Internet clickstreams from thousands of high-traffic Web sites with thousands of third-party lists, databases, and clickstreams purchased from data exchanges. Interclick's proprietary analyses decipher customer intent and probable lift (meaning, improvement in performance) for ad campaigns.
Someone known to have visited a car-configuration app on an auto manufacturer's Web site, for example, is a good bet to be in the market for a car. And someone who browsed smart phones available from a particular phone maker, carrier, or electronics retailer is likely to be ready to purchase a new phone.
These are simplistic examples and just two of more than 2,000 audience segments that Interclick continuously tracks and targets across its network of several thousand top-traffic Web sites. The members of each segment are anonymous (meaning they're not targetable by name). But using tracking technologies such a cookies, Interclick can serve up adds to each individual -- be it a probable pickup truck buyer, a probable smart phone buyer, or 2,000-plus other segmention members -- when they visit one of Interclick's network-affiliated sites.
The OSM service is built on ParAccel, a database that combines massively parallel processing on commodity hardware, a columnar architecture for fast querying and efficient data compression, and ultra-fast in-memory support as required. The company settled on ParAccel in early 2009 after also considering Aster Data, Greenplum, Netezza, and Vertica. Among its requirements, Interclick has single queries that operate on tables with 90-plus billion records, and they sometimes require multiple passes against the same database tables.
"Those queries didn't work well on most of the products we considered," says Andrew Katz, Interclick's co-founder and chief technology officer. Katz says ParAccel's columnar capabilities and query-plan generation seemed to make a difference for the company's specific workloads -- a testament to the importance of testing your own data and queries before buying such a platform. (Interestingly, ParAccel is now one of the few independents remaining while the four competitors mentioned above have since been acquired by Teradata, EMC, IBM, and HP, respectively.)
Development of the overall OSM system required months of proprietary development work, but its final deployment on the ParAccel database took only about three months, according to Katz. Upon launch, OSM served up more than 1,000 customer segmentations, and since then the company has steadily added more targeting information and audience segments.
In a "look-alike" analysis added in 2010, for example, Interclick uses a blend of eight scoring techniques to identify audiences likely to behave like known groups such as car purchasers. "If we can find people who share eight to 10 attributes with people who recently purchased cars, that tells us they are more likely to buy a car," Katz explains.