Inside An Ad Agency's Big Data Nerve Center

VivaKi, a global advertising agency, is integrating the Kognitio Cloud analytical platform into its research and development data hub, a move that will dramatically reduce the time it takes to query very large data sets, the company said.

The R&D hub, known as the VivaKi Nerve Center, is a resource shared by the company's four major agency groups--Digitas, Razorfish, Starcom Mediavest Group, and ZenithOptimedia--as well as its business clients and partners.

"It's basically a collection of all the technology assets that support these digital advertising companies," said Pradeep Ananthapadmanabhan, CTO for the VivaKi Nerve Center, in a phone interview with InformationWeek.

Kognitio is an in-memory analytical platform that organizations use to accelerate their business intelligence (BI) or data management system. The cloud version, aptly named Kognitio Cloud, is a platform-as-a-service (PaaS) that makes Kognitio's analytical platform available on a subscription basis. It provides an open database connectivity (ODBC) system for managing and analyzing data. For IT departments, the cloud-based analytics platform eliminates in-house maintenance tasks, including software and hardware upgrades.

[ Don't get sidetracked from the start. See Big Data Projects: 6 Ways To Start Smart. ]

The VivaKi Nerve Center will deploy Kognitio Cloud through Amazon Web Services. "We use a suite of cloud services from Amazon, including other big data processing technologies like Hadoop," said Ananthapadmanabhan.

For VivaKi, one of Kognitio Cloud's best attributes is its ability to turn massive amounts of data into analytical queries, and do so at speeds that haven't been possible with relational databases. The company had previously been using traditional tools such as SQL server to process terabytes of data from multiple sources, including Google Search.

In recent tests with Kognitio Cloud, VivaKi has been able to slash the amount of time it takes to process massive data sets from up to 10 hours to just 30 minutes. And with testing now complete, the VivaKi Nerve Center plans to go live with Kognito Cloud very soon.

"Advertising specifically can benefit by leveraging big data technologies," said Ananthapadmanabhan. Kognitio's technology has enabled VivaKi to add extra analytical capability in under a week, rather than the months required with traditional data management systems, he added.

In-memory analytical platforms are generating a lot of buzz these days, a development that's likely to draw attention to Kognitio and its competing services. According to Gartner's 2012 Hype Cycle Special Report, big data terms such as "private cloud computing" and "in-memory database management systems" are currently at their "peak of inflated expectations."

On the flip side, resistance within some industries may hamper the adoption of new data-management tools. A recent report by Quartet FS, maker of the ActivePivot in-memory analytics platform, shows that 57% of IT managers see the "newness" of in-memory analytics technology as the "biggest barrier to adoption at their bank."

Any reticence on the part of IT may be short-lived, however, particularly if new technologies prove effective at solving big data-related problems. Research firm IDC predicts the market for big data technology and services will soar from $3.2 billion in 2010 to $16.9 billion in 2015--a compound annual growth rate (CAGR) of 40%.

And Forrester predicts the adoption of public and private cloud solutions for BI will rise quickly in the near future. In fact, 38% of companies surveyed recently by Forrester say they're planning a BI software-as-a-service project before the end of 2013.

"Information is exploding all around us: 1,500 blogs, 98,000 tweets, and 168 million emails every minute just to mention a few of the many sources that contribute to the tremendous data growth," wrote Forrester analyst Holger Kisker in a recent blog post. "This year we will hit a volume of 2.7 zettabytes of global digital data and Forrester predicts that ongoing data growth will outperform Moore's Law over the next few years."

Expertise, automation, and silo busting are all required, say early adopters of private clouds. Also in the new, all-digital Private Clouds: Vision Vs. Reality issue of InformationWeek: How to choose between OpenStack and CloudStack for your private cloud. (Free with registration.)