Software // Information Management
Commentary
11/24/2010
11:04 AM
Doug Henschen
Doug Henschen
Commentary
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

ComScore's Big Data Deployment In Detail

Crunching tens of terabytes, a leading digital marketing measurement firm delivers insight -- quickly.

When your slogan is "we measure the digital world," you better have serious data-crunching capabilities to back up your claims.

comScore has been delivering big-data intelligence for ten years, and a recent refresh of its data warehousing environment is expected to take the company well into another decade.

Read the stream of press releases churned out by comScore and you'll quickly understand its business. One day it might report on the top-50 Web sites on the Internet. The next it might divulge the market shares of the Internet's leading search engines.

comScore reports on markets across the globe, detailing, for example, the number of mobile phone users in the top-five EU countries receiving SMS-based advertisements -- answer: more than 100 million.

In short, comScore sells data, and the faster it can collect and crunch high-volume samples of Internet and mobile-device usage data, the more numerous and valuable the company's insights become.

"If we can sell last-month's data this month, it has a certain value, but if we can sell last week's data this week, it has even more value," explains Scott Smith, comScore's vice president of data warehousing.

To enhance that turnaround time, Smith has overseen multiple refreshes of comScore's data warehousing platform over the last 10 years. All of them have run on the Sybase IQ column-store database.

comScore's first deployment, way back in 2000, was built on eight Dell severs. The latest, installed in March 2010, runs on 12 Dell R710 servers and an EMC storage area network (SAN).

"I've gone through almost the entire Dell rack server line, from the old 6800s to today's R710s," says Smith. "We try to keep it to $10,000 a pop so you can bring another [database] reader or writer onto a node."

comScore had 56 terabytes of compressed user data on its old deployment while the new system can handle up to 150 terabytes with plenty of room to grow. Smith says he's already planning to add 10 more R710 servers for processing power, and the SAN can grow as needed. Sybase IQ's Multiplex grid lets comScore scale up processing power and storage incrementally and independently.

Previous
1 of 2
Next
Comment  | 
Print  | 
More Insights
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest, Nov. 10, 2014
Just 30% of respondents to our new survey say their companies are very or extremely effective at identifying critical data and analyzing it to make decisions, down from 42% in 2013. What gives?
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of November 16, 2014.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.