Software // Information Management
11:04 AM
Doug Henschen
Doug Henschen
Connect Directly
How to Keep Mobile Threats at Bay
Jun 01, 2016
With savvy cybercriminals using vulnerabilities in apps, networks, and operating systems to gain c ...Read More>>

ComScore's Big Data Deployment In Detail

Crunching tens of terabytes, a leading digital marketing measurement firm delivers insight -- quickly.

"If I had gone with the big Unix-box approach, at some point there would be a major cost," he explains. "We went with the commodity approach because we like buying things with five zeros rather than six zeros."

Compression is a key advantage for column-store databases. comScore gets anywhere between 25% and 40% compression, depending on the type of data being analyzed, according to Smith. In terms of performance, increases in processing power and data input/output speed and throughput have brought steady improvements. As an example, an analysis of Internet search engine traffic that took five hours on comScore's last Sybase IQ deployment now takes 11 minutes in the new deployment, Smith says.

Sybase IQ runs on symmetric multiprocessor (SMP) hardware, a contrast with the many appliances and databases that now run on massively parallel processing (MPP) architectures. MPP architectures have advantages when it comes to scalability, but Smith says Sybase IQ's Multiplex (grid scaling) approach has advantages in terms of flexibility.

"I can do parallelized loads and the things that MPP is good at, but I also have the throttle of being able to do multiple tasks at the same time with Multiplex," he says, contrasting the multiple, independent database nodes of the IQ grid with the single-database-controller/worker-node approach of MPP.

Smith says comScore has looked at MPP alternatives from the likes of Vertica and ParAccel, column-store competitors to Sybase IQ, but a switch would also mean the company couldn't leverage prior investments.

"I have all this commodity hardware that I can just move over to the new warehouse, so why wouldn't I do that?" he says.

In the never-ending challenge of analyzing more data and doing it more quickly, Smith says Sybase IQ has always provided what he describes as an extensible model.

"We don't want to design into a box, so I'm always interested in where Sybase is planning to go and how I'm going to leverage the platform."

Incremental investment, the ability to leverage existing hardware, extensibility when it comes to processing power and storage: these aren't capabilities that are exclusive to Sybase IQ, but not all data warehousing approaches and technologies give you this flexibility. Thus, take a cue from comScore's experience and explore these system capabilities in detail in your next data warehousing RFP.

2 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
2016 InformationWeek Elite 100
Our 28th annual ranking of the leading US users of business technology.
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.