InformationWeek: The Business Value of Technology

Lesson 7: In Many Industries, Speed Trumps Scale

10 Lessons Learned By Big Data Pioneers

Start The Discussion | Doug Henschen | August 23, 2011 14:10 PM

How can you prepare for the big data era? Consider this expert advice from IT pros who have wrestled with the thorny problems, including data growth and unconventional data.

Informationweek Discussions

To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.

Previous Slide Next Slide 8 of 12

Lesson 7: In Many Industries, Speed Trumps Scale

In many scenarios, the "big" in big-data isn't the ultimate scale of the database so much as the amount of information loaded and analyzed each day. Marketers, for example, typically need to load and analyze lots of data as quickly as possible so the insight can be quickly applied to identify new segments and lists, and to improve targeting or creative content for the next campaign. If you know what's working sooner, you won't waste money on segments, enrichment data, or marketing appeals that aren't fruitful. To provide search, lead-generation and affiliate marketing services to publishers and advertisers, ad network Linkshare loads and analyzes tens of gigabytes of clickstream data per day, but the total database tops out at just six terabytes. Low-latency insight is now a competitive must for Linkshare. "Five years ago it was okay to give people yesterday's data, but that's not good enough any longer," says Jonathan Levine, LinkShare's chief operating officer.

RECOMMENDED READING

Big Data A Big Backup Challenge

Big Data: Informatica Tackles The High-Velocity Problem

IBM Picks Hadoop To Analyze Large Data Volumes

Hadoop Big Data Startup Spins Out Of Yahoo

2 Ways Big-Data Analysis Pays Off

A Model For The Big Data Era

Machines Are Driving The Big-Data Era

NOAA CIO Tackles Big Data