Big Data's Surprising Uses: From Lady Gaga To CIA

Cisco's 2012 Visual Networking Index (VNI) Forecast predicts that nearly 19 billion network connections will be operational by 2016 -- nearly 2.5 connections for every human on earth -- up from 10.3 billion in 2011. Not surprisingly, the rising popularity of mobile devices, specifically smartphones and tablets, is driving that explosive growth.

Billions of mobile devices generate enormous volumes of data. In fact, Cisco estimates that global IP traffic will be 1.3 zettabytes within three years. (One zettabyte equals 1,099,511,627,776 gigabytes.) By any definition, that qualifies as big data.

But many retailers aren't prepared to manage this explosion in mobile-generated information, argues Brian Lent, cofounder and chief technical officer of Medio, a provider of predictive analytics products for the mobile industry. Before launching Medio in 2004, Lent served as director of information technology at Amazon.com, and also started three other venture-backed startups.

[ Data from mobile health apps might soon be helping heart patients. Read UCSF Heart Disease Study To Use Mobile Apps. ]

Without real-time analytics and data scientists, big data is just a storehouse of dusty bits, Lent told InformationWeek in a phone interview. "You can collect and store a lot of data efficiently," he said. "But if you don't know how to take that data, do analytics on it, and put it into operational use, you don't get the real value out of it."

Retailers need to develop predictive models to identify and monetize mobile customers, said Lent, as well as develop effective promotions and incentives to retain customers and reduce churn. He agrees with the popular assertion that data is the oil of the 21st century. "That's a premise that I've lived and breathed for a couple of decades," he said. "I can't think of any better industry than mobile as to where that's epitomized."

And with mobile shopping emerging as the next evolution in commerce, retailers must focus on collecting and analyzing a valuable new set of data. "Technologies like Hadoop, which are clearly in the big data space, are critical," he said. "But it's the real-time component -- in our case, with predictive analytics -- that makes big data actionable."

Retailers, he noted, should take several steps to achieve this, such as designing mobile tools, including Android, iOS, and HTML 5 apps, that enable rich data collection.

"One of the most valuable lessons I've learned over my career is that if you don't log each and every click or event, you can't reinvent that information," Lent said. "So make sure you have a very comprehensive way to instrument your apps with a simple SDK, (and then) log the data and pull it into a big data structure."

The right set of predictive models can help retailers retain customers, in part by allowing them to deliver relevant ads, offers, and promotions, he noted. "The only way to do that in a balanced fashion is with what we call closed loop optimization, meaning that as the user clicks on the mobile app, and then the data is logged, we produce a recommendation," he said.

Customer information and predictive analytics are only two-thirds of the data science experience, said Lent. The third component: human experts. "We call them the 'pink unicorns' of the industry. These are the data scientists, the predictive analytic scientists. They're the ones, in a given industry like retail, who need to constantly adopt new technologies and tune analytics for that industry," he said.

Lent sees big data become more of a marketing matter than a technical issue, one that falls under the purview of a corporation's chief marketing officer (CMO) rather than its chief information officer (CIO).

"The CMO is going to be spending more IT dollars because the CMO, in marketing areas, is driving a lot of the demands for big data," he said.

Companies want more than they're getting today from big data analytics. But small and big vendors are working to solve the key problems. Also in the new, all-digital Analytics Wish List issue of InformationWeek: Jay Parikh, the Facebook's infrastructure VP, discusses the company's big data plans. (Free registration required.)