Big Data // Big Data Analytics
12:14 PM
Connect Directly
Repost This

Big Data 101: New Vendor-Neutral Guide

Open Data Center Alliance offers a big data primer for enterprises -- and sounds a call to action for standards-based big data technologies.

Big Data's Surprising Uses: From Lady Gaga To CIA
Big Data's Surprising Uses: From Lady Gaga To CIA
(click image for larger view and for slideshow)

What exactly is big data? Explanations vary, of course, but Gartner's popular 3V model -- essentially that big data is high-volume, high-velocity, and high-variety information that requires new tools to manage -- is the most commonly quoted definition. Still, a haze of ambiguity surrounds big data, an issue that the Open Data Center Alliance (ODCA) hopes to resolve with its new "Big Data Consumer Guide," a handbook for enterprises that want to know what big data is, why it matters, and how to gain from it.

Created in 2010, the ODCA is an industry consortium of global IT organizations with a primary goal of developing open standards for cloud computing. In a phone interview with InformationWeek, ODCA executives said that the alliance's "Big Data Consumer Guide" is a logical extension of its cloud computing efforts. "A lot of the cloud paradigms out there today are something that the big data environment is going to want to take advantage of," said ODCA lead technical advisor John Pereira.

[ Big data has value that's often not reflected in the books. Read What's Your Big Data Worth? ]

The essential nature of big data -- particularly the fact that data volume can grow significantly in a short period of time with little notice -- fits nicely with the cloud environment, Pereira noted.

"Because of the nature of big data, you're probably going to want to think of something in a more distributed environment. A cloud paradigm helps you move in that direction," he added.

The consumer guide summarizes how big data platforms can help a variety of industries. Banks, for instance, can correlate data from multiple, unrelated sources to potentially spot credit card fraud. In addition, the guide provides common definitions and lingo that organizations can use when working with big data providers.

The guide also references an astonishing statistic from IDC: Unstructured data comprises more than 90% of the information in today's enterprises, much of it stored away in documents, email, notes and Web content.

Unstructured information that falls under the "big data" umbrella includes machine-generated data from sensors, machine logs and cellphone GPS signals, as well as data from social media sites and online transactions.

The Consumer Guide refers to Apache Hadoop as a "leading Big Data technology," but points out that a variety of other open-source big data projects are available, including Riak, MongoDB, CouchDB, Redis, Hypertable, Storm, Spark and High-Performance Computing Cluster (HPCC).

"We really try to bring a relatively vendor-agnostic approach to our recommendations and direction," said Pereira. "We try to avoid beating the drum for a specific vendor."

Enterprises need to carefully plan their big data strategies in advance to avoid poor practices that waste both resources and money, the ODCA says.

"You want to be able to write the data in a way that is most efficient -- not replicate the same data set over and over. How you record the information up front is very important," said ODCA executive director Marvin Wheeler. "It's all about how the data's being written so that it doesn't sprawl as much as (with) traditional methods." Dealing with data sprawl is already a critical issue for enterprises. According to the McKinsey Global Institute, 15 of 17 U.S. business sectors have more data stored per company than the U.S. Library of Congress. And some researchers estimate that 90% of all data has been generated in the last two years alone.

The growing use of video analytics is one example.

"If you go back five years ago, who would have thought that the idea of keeping video -- and being able to do analytics on it to make potentially better business and shopping decisions -- was a topic that would become foremost in everybody's mind," said Pereira. "It does get to the heart of what big data is about, and it's one of the things that makes big data very interesting as a new technology and paradigm."

Emerging database technology promises to automate more analysis. Here's where it could replace relational systems. Also in the new, all-digital The Rise Of Semantic Databases special issue of InformationWeek: There's a big demand for big data and analytics experts. (Free registration required.)

Comment  | 
Print  | 
More Insights
InformationWeek Elite 100
InformationWeek Elite 100
Our data shows these innovators using digital technology in two key areas: providing better products and cutting costs. Almost half of them expect to introduce a new IT-led product this year, and 46% are using technology to make business processes more efficient.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.