6 Ways To Build Trust In Big Data

With big data, the biggest hurdle isn't so much achieving success. It's getting people to believe in big data -- to trust it. And it's not just trusting the data itself. It's trusting what can be done with the data.

A single piece of data has only so much intrinsic value. What contributes the most value to business are the correlations done to provide insight, perspective and meaning to a given data point. However, it's not easy to get people to trust insights pulled from models based on big data. Do we trust our gut, our experience, our intuition -- or the data?

Even when data correlations show a boost in sales or efficiency, business leaders may not believe what they see -- at first. People will not want to give up the gut-level decision-making processes they've used for years until they are convinced beyond a shadow of a doubt that big data works. Proving big data's worth may take more than one successful project.

[ Are you focusing on the right data? Read Big Data: Know What You Want. ]

Earning trust also requires tact as you interact with business leaders whose turf you are treading on. For the past three years, Intel has been implementing big data initiatives internally with great success. But we've had to go through trial and error to get where we are today. A major part of the initiatives involves gaining the trust of key stakeholders within the business whose problems we attempt to solve.

Through this process, we've identified six steps toward earning trust in big data:

1. Understand the business and understand the data. It may seem obvious, but undertaking an often complex analytics deep-dive for a line of business requires sitting down with key people to understand what that line does, how it interacts with the rest of the company, and its challenges. What is impeding progress? What is keeping them from being more efficient? You'll need a business process thinker for this -- someone who can ask the right questions and has a good understanding of the available data.

2. Determine the problem and how the data can help. Start to connect the dots between the business problem and the available data. Will this data help solve this problem? At this point you might realize that the type of data you need is missing. Can you get access to it?

One caveat: People tend to think of big data as social media and the Internet of Things (IoT). They feel the need to immediately go outside of the enterprise to mine that type of data, and sometimes that is necessary. But integrating outside data adds complexity. I would argue that there is a significant amount of untapped value in the data inside the organization. Start there and determine whether pulling in external data is necessary or helpful.

3. Set reasonable expectations -- walk away if you have to. Make sure the business understands that for every business problem solved, there may be three or four unsolved. We've spent a couple of months on some projects that didn't yield enough significant value.

For example, we worked with our sales and marketing team to better understand how internal market forecast information compared to the external industry forecasts. We pulled in several external data sources and blended it with our own. After several months, we found the quality and context of the different forecasts was not effective enough to include so we moved the project to the back burner. Now we have a better understanding of how to carefully assess the viability of external data sources before we head too far down the solution path.

4. Bring in big data while living in the old world. Approach big data projects in parallel with traditional methods such as business intelligence tools or people's own historical business gut feel. Business leaders are not going to give up familiar methods and say, "Okay, I'll trust the data now." You've got to prove it to them while still operating within their parameters for making decisions.

5. Be flexible. The big data analysis you're embarking on is an exploration of sorts. You may find value in an unexpected area. For example, Intel embarked on exploring reseller buying patterns to identify the top performing resellers. When we started correlating resellers across regions, interesting patterns emerged that enabled us to recommend new products and identify new resellers to target. Hence you need to be flexible about methodology and tools. Big data tools are in the early stages so your toolset will probably not be the same in a year. You need to be flexible in implementing tools, upgrading and investing as needed.

6. Keep your eye on the prize. The process will feel cumbersome at times. It's in those moments that you need to stay focused on results. The methods and tools for easy, efficient data analysis aren't quite there yet. But at Intel, using the latest big data tools we analyzed our historical test data, which helped us to select the validation methodologies that returned the most value for the time invested. The result was a cut in our validation time by 25%, which helps us to get products to market faster. Results like this make the whole process worthwhile.

Building trust in big data insights takes time. It could take up to six months just to prove that first business case -- as it did in our case. But once a few initiatives are under your belt, the business will warm up to big data and will be clamoring for more. Not only will the business change, but IT will be transformed into a strategic and trusted business partner.

The big data market is not just about technologies and platforms -- it's about creating new opportunities and solving problems. The Big Data Conference provides three days of comprehensive content for business and technology professionals seeking to capitalize on the boom in data volume, variety and velocity. The Big Data Conference happens in Chicago, Oct. 21-23.