2 Ways Big-Data Analysis Pays Off

Interclick's experience shows that getting answers within seconds can improve productivity, but sometimes it's also the only way to reach best prospects before they buy.
Interclick also developed a lift-analysis technique whereby it can test the influence of any given attribute on a test population and predict the improvement in performance it might generate. The tool then identifies large groups of network users who share that attribute.

So what's an example of an attribute? It could be a non-obvious association. For example, a car manufacturer planning a campaign for a pickup truck would seek an audience with obvious brand affinities, such as males between the ages of X and Y who are known to be sports enthusiasts. Lift analysis performed in OSM might reveal that best prospects also happen to be parents with children of a certain age who drink a certain sports drink and are Android users rather than iPhone users, says Katz, offering a theoretical example.

These are productivity scenarios. Interclick has time and capacity to find look-alikes and do lift analysis because the company's analysts aren't waiting around for more basic analyses to be performed.

More recently, however, there has been a push for more near-real-time analysis.

In preparation for a new service the company is planning, Interclick recently converted a conventional hard-disk-based cluster to a pure in-memory cluster running on 3.2 terabytes of RAM. The goal was to get at least a 10-times improvement in performance, and Katz says the deployment surpassed that goal.

The bottom line is that the in-memory cluster has cut certain queries from hours down to minutes or seconds, and it's the kind of performance required to deliver what Katz calls "short-tail, intent-based campaigns," meaning the segmentation has a short, but valuable, shelf life.

Would-be business travelers are a case in point. Interclick has only a matter of hours, perhaps even minutes, in which it can serve up ads to people who have demonstrated an interest in business travel before they have booked their flights, rental cars, and hotel rooms. To do so it has to be able to ingest timely behavioral data, analyze it, identify a segment, publish a campaign offer and deliver a targeted campaign all within minutes.

"Going from hours on segment creation and publishing down to minutes is a huge win for us," Katz says.

There are, of course, more direct and obvious ways to reach best prospects. Travel-related sites, tech-review sites, and automotive sites can obviously serve up audience-relevant offers as people are booking flights, shopping for cell phones, or browsing cars, but the audience numbers tend to be small and ad placements pricey. Ad networks crunch big data to spot big audiences that can be segmented in many ways. Costs increase along with the precision and timeliness of the targeting data and the value of the transaction -- the old recency-frequency-monetary value rule of thumb still applies.

There are many near-real-time scenarios outside of advertising and marketing circles, but you don't often hear the details. In some cases they're trade secrets or even a matter of national security. In security circles, for example, big-data analysis spots malware, viruses, and hacking attempts. That's information that has to be acted upon quickly. And in the intelligence community, analyses of network traffic, voice data, and messaging is undoubtedly going on behind the scenes.

Delivering ads in a timely way may sound a bit less glamorous than thwarting hackers, spies, or terrorists, but it's good example of the kind of near-real-time action on insight that's now possible in the big-data era.

Editor's Choice
James M. Connolly, Contributing Editor and Writer
Carrie Pallardy, Contributing Reporter
Shane Snider, Senior Writer, InformationWeek
Sara Peters, Editor-in-Chief, InformationWeek / Network Computing
Brandon Taylor, Digital Editorial Program Manager
Jessica Davis, Senior Editor
John Edwards, Technology Journalist & Author