2 Ways Big-Data Analysis Pays Off - InformationWeek
Software // Information Management
09:56 PM
Doug Henschen
Doug Henschen
Connect Directly

2 Ways Big-Data Analysis Pays Off

Interclick's experience shows that getting answers within seconds can improve productivity, but sometimes it's also the only way to reach best prospects before they buy.

No doubt you've read about the advantages of modern, big-data analysis platforms. Queries that once took days or hours can be cut down to minutes or seconds. So what's the big hurry? Ad services company Interclick’s use of the platform illustrates the two key ways companies benefit from fast, big-data analysis.

Lots of people talk about the need for speed, but you don't often hear about scenarios in which insights are acted upon within minutes of their discovery. Let's call that a near-real-time scenario. Much more common are examples in which fast querying frees up time for more analysis -- either exploring more data or doing deeper queries. Let's call that a productivity scenario.

For Interclick, a fast-growing New York-based firm that strives to put the right online ads in front of the right people at the right time, both scenarios come into play. The company launched its core Open Segment Manager (OSM) platform in 2009. OSM is essentially a data-driven service that helps advertisers reach targeted segments of would-be customers. The database combines Internet clickstreams from thousands of high-traffic Web sites with thousands of third-party lists, databases, and clickstreams purchased from data exchanges. Interclick's proprietary analyses decipher customer intent and probable lift (meaning, improvement in performance) for ad campaigns.

Someone known to have visited a car-configuration app on an auto manufacturer's Web site, for example, is a good bet to be in the market for a car. And someone who browsed smart phones available from a particular phone maker, carrier, or electronics retailer is likely to be ready to purchase a new phone.

These are simplistic examples and just two of more than 2,000 audience segments that Interclick continuously tracks and targets across its network of several thousand top-traffic Web sites. The members of each segment are anonymous (meaning they're not targetable by name). But using tracking technologies such a cookies, Interclick can serve up adds to each individual -- be it a probable pickup truck buyer, a probable smart phone buyer, or 2,000-plus other segmention members -- when they visit one of Interclick's network-affiliated sites.

The OSM service is built on ParAccel, a database that combines massively parallel processing on commodity hardware, a columnar architecture for fast querying and efficient data compression, and ultra-fast in-memory support as required. The company settled on ParAccel in early 2009 after also considering Aster Data, Greenplum, Netezza, and Vertica. Among its requirements, Interclick has single queries that operate on tables with 90-plus billion records, and they sometimes require multiple passes against the same database tables.

"Those queries didn't work well on most of the products we considered," says Andrew Katz, Interclick's co-founder and chief technology officer. Katz says ParAccel's columnar capabilities and query-plan generation seemed to make a difference for the company's specific workloads -- a testament to the importance of testing your own data and queries before buying such a platform. (Interestingly, ParAccel is now one of the few independents remaining while the four competitors mentioned above have since been acquired by Teradata, EMC, IBM, and HP, respectively.)

Development of the overall OSM system required months of proprietary development work, but its final deployment on the ParAccel database took only about three months, according to Katz. Upon launch, OSM served up more than 1,000 customer segmentations, and since then the company has steadily added more targeting information and audience segments.

In a "look-alike" analysis added in 2010, for example, Interclick uses a blend of eight scoring techniques to identify audiences likely to behave like known groups such as car purchasers. "If we can find people who share eight to 10 attributes with people who recently purchased cars, that tells us they are more likely to buy a car," Katz explains.

1 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
9/11/2012 | 8:44:42 PM
re: 2 Ways Big-Data Analysis Pays Off
Great article.
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
Digital Transformation Myths & Truths
Transformation is on every IT organization's to-do list, but effectively transforming IT means a major shift in technology as well as business models and culture. In this IT Trend Report, we examine some of the misconceptions of digital transformation and look at steps you can take to succeed technically and culturally.
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll