Big Data: Know What You Want

When the term "big data" stormed into the tech lexicon a few years ago, I thought, "Well, that really narrows it down." "Data" is a vague word, with another vague word -- big -- tacked on.

I had visions of endless rows of giant haystacks. Now go find the needles.

But its daunting name notwithstanding, big data is not about data piles; it's about data analytics. You need to know what insights to glean from the haystacks and how often you need them, and you and your important business leaders must agree that you're focused on the right data. You need to present it to your employees/managers/executives in a simple and clear way. Then you may just find that magic edge over the competition by predicting -- and reacting to -- customer needs or sales trends.

Suddenly, big data isn't so vague.

[ Are you using big data to make better decisions? Read Big Data's Holy Grail: Better, Faster Decisions. ]

You don't have to look far to find early big data insights and success stories. In the travel industry, airlines, hotels and rental car agencies are applying big data analytics to pricing, inventory and customer habits. In the auto industry, big data and cloud computing promise safer roads through tailored GPS updates and on-board diagnostic sensors that alert the car dealer that you need repair and are on your way. Heck, big data analytics can even tell a woman when she is most fertile.

But despite big data's promise, our research shows that companies are struggling to exploit big data analytics for practical business use.

Just 9% of the 257 respondents to our InformationWeek 2013 Big Data Survey rate their organizations as extremely effective at identifying critical data and using it to make decisions.

In a separate InformationWeek big data research report from March 2013, only 16% of the 417 business technology pros at companies using or planning to deploy data analytics software said they have no concerns about big data analytics.

Here are the top three big data concerns:

-- Big data expertise is scarce and expensive (38%)

-- Data warehouse appliance platforms are expensive (33%)

-- Unsure of how big data analytics will create business opportunities (31%)

With big data, many organizations try to run before they can walk. Because of competitive pressure (and relentless hype), investments are often rushed without careful planning and consultation with all business units, and the results can be detrimental. The impact of sloppy big data investments is outlined in detail in this column. The bottom line: Big data investment decisions are best made by the entire company.

Finding order -- and business opportunities -- in chaos is the subject of a recent InformationWeek feature story: The Data That Matters.

A key takeaway from the story is that presenting data is as important as finding it, and that's where data visualization tools come in. Given that 58% of the 517 business technology pros cited "access to relevant, timely or reliable data" as the main roadblock to information management success, companies need to make data relatable and visual.

Data visualization tools create heat maps, data relationship maps and trees, and geospatial overlays. They provide clear, colorful and visual ways to explain a sales trend to the CEO in a few minutes. They turn data into a conversation.

These tools are not widely adopted yet (although you can expect that to change), and in the wrong hands, they can produce flashy but inaccurate data sets. But when it comes to simplifying and exploring data, data visualization should be part of any big data strategy worth its salt.

What's the end game for your company's big data strategy? Do the different business units and the CEO agree on it? Don't go near those data haystacks until you've identified a clear company-wide objective.

The big data market is not just about technologies and platforms -- it's about creating new opportunities and solving problems. The Big Data Conference provides three days of comprehensive content for business and technology professionals seeking to capitalize on the boom in data volume, variety and velocity. The Big Data Conference happens in Chicago, Oct. 21-23.