6 Steps To Manage Big Data

Big Data Talent War: 7 Ways To Win

(click image for larger view and for slideshow)

The volume, velocity and variety of big data are growing exponentially, but which of these three factors presents the biggest challenge for organizations? The winner is variety, meaning data that comes from a wide range of different sources, according to Coveo, a Quebec City, Canada-based company that sells advanced search technology to enterprises.

"Companies understand there's a lot of data out there, and it could potentially create value for them, but they don't really know where to start," said Diane Berry, Coveo senior VP of marketing and communications, in a phone interview with InformationWeek.

Berry and Coveo's engineers offer the following six ways to manage this motley mix of data, which often comes from sources outside an organization, including social networks and online communities.

1. Understand Your Business Goals Beforehand.

Unstructured data from customers -- particularly from social media -- can be overwhelming and of limited value if you don't know how to use it effectively. Even if companies know they have a data management problem, they may lack clearly defined goals on how to solve it.

2. Don't Try To Move The Data.

Let the information stay where it is. Don't attempt to move unstructured data to your organization's system of record, such as CRM or knowledge base. "It's very difficult to move data into a single system and have it remain relevant and up to date," Berry said.

[ How can universities help meet the growing demand for data scientists? Big Data Education: 3 Steps Universities Must Take. ]

If unstructured data is unavailable to users while it's being moved, for instance, it may be out of date when it is available. In addition, large-scale projects to integrate disparate data systems can be costly, take years and can cause headaches for IT teams.

3. Replicate And Duplicate Data? Don't Do It.

Users love to duplicate data on their desktops, particularly if they've had a tough time finding the information they need elsewhere. The problem with this approach is that the user probably doesn't update this data, which grows old and fragmented.

"It's a very common problem. It's human nature. If it's information that you use all the time, and it's difficult to find, you're going to copy that information and put it somewhere where you know you can find it," said Berry.

4. Use Indexing Technology To Navigate A Complex Data Environment.

An enterprise data system is heterogeneous, pulling in information from a variety of sources. Many companies have enterprise search; but if users don't get the results they want they'll stop using the system. But a real-time, central, unified index can help organizations manage this complex environment.

"Having this highly advanced, unified index provides much more benefit because you're going to have information that's always up to date, and also related to other information from other systems," Berry said.

5. Present Information Based On User Actions.

Advanced indexing technology unifies data from many sources, and presents information that's relevant to the user. For customer service reps, for instance, the ability to run text analytics across different channels -- such as online chat, phone calls, email and even Twitter -- enables them to have a handle on everything a customer has said concerning a particular issue.

6. Spin Data Into Insights.

Information must drive a return on investment. If an organization can tap into data's economic value, it can boost innovation, better serve its customers and get an edge on the competition, Coveo said.

Companies must find ways to understand the expertise that exists both in-house and outside their organizations, and how to share that knowledge with their employees.

Unified indexing technology, Berry believes, can help global companies communicate better.

"Understanding what an engineer across the world is working on might help another engineer in a different office, or help in sales or customer support," she said.

Predictive analysis is getting faster, more accurate and more accessible. Combined with big data, it's driving a new age of experiments. Also in the new, all-digital Advanced Analytics issue of InformationWeek: Are project management offices a waste of money? (Free registration required.)