Software // Information Management
01:00 PM
Connect Directly
Core System Testing: How to Achieve Success
Oct 06, 2016
Property and Casualty Insurers have been investing in modernizing their core systems to provide fl ...Read More>>

6 Lies About Big Data

Our 2013 Big Data Survey shows we're not lacking facts, figures, or tools to wrangle them. So why do just 9% of respondents rate themselves as extremely effective users of data?

Lie 5: We know what data we need

In our survey, we asked about 10 internal and nine external data types. Internal sources include financial accounting applications, detailed sales and product data, CRM data, unstructured network data such as Office files and images, and unstructured data stored on end user devices. External sources include government statistics and other public records, geolocation data, data collected from sensors on company products and services, social network data (Facebook, Twitter), and unstructured data stored in the cloud (Office365, Google Docs).

Clearly, there's a lot of information out there. But when we asked who's driving data analysis ideas, we were surprised to find that only 5% of respondents have a centralized team to drive big data strategy; an additional 3% use a looser collaborative effort.

We're not the biggest fans of committees, but given the fact that the users of your data are likely spread far and wide, it makes sense to create a cross-functional group to identify new sources or elevate the importance of an existing stream. It's staggering to see some of the great data that's all but untouched.

Take CRM, phone, email, and Web analytics. These four data points cover most of the communications relationship with your clients. Tying them together isn't rocket science, especially if you have decent baseline customer data to start with. Not only can you determine the number of conversations your company typically has with customers, but you can also understand how email relates to phone calls and Web traffic. If you have an outside sales force looping in, your CRM data gives you a profiling capability to model everything from product rollouts to customer service problems.

All that intelligence exists today, yet few companies have this level of analysis integrated into their big data strategies. While 35% of survey respondents say their IT organizations include CRM in their integrated plans, only 29% include email, 22% Web analytics, and 14% phone logs.

Lie 6: We do something with our analysis

There's nothing more frustrating for an analyst than to work for days or weeks on a project, present the findings, have a great meeting with execs, then watch those recommendations die on the vine. Everyone focuses on the positive aspects of data analysis--helping find new customers or discover more productive logistics routes. But the reality is that big data analysis will find some negative things--about your sales team's effectiveness, your online presence, your true costs of operations. The slow economy of the last four years has weakened multiple parts of most companies. Adding data sources and a more holistic analysis will help find and prioritize the problems you need to fix.

IT Truth Tellers

Want to raise IT's profile as a business enabler? Step in and assume responsibility for data quality across the company. Here's a quick check of items IT should review today:

1. Is there a centralized data quality team? If not, set one up ASAP.

2. Does the team do regular or, at minimum, spot audits of various analyses? Does it regularly look to add new data sources?

3. Are critical external events annotated within your data warehouse or as part of your reporting process? For example, think about major system upgrades that would change the underlying data related to order flow.

4. Do you require statistical notes, including sampling statements?

5. When it comes to customer or vendor surveys, are sample sizes validated against your base customers or total market size to ensure accuracy?

6. Do you run regular "stress tests" for current data sets with cross-functional teams, challenging assumptions and sacred cows?

7. Do you look outward? Most respondents, 75%, report some public cloud use. Yet many companies aren't capturing associated data--think WebEx conferencing for customer behavior analysis or Google analytics for sales tracking.

chart: How effective is your company at identifying critical data and using it to make decisions?

3 of 3
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Engineer Veteran
Engineer Veteran,
User Rank: Apprentice
11/13/2012 | 2:06:29 PM
re: 6 Lies About Big Data
I couldn't agree more. There is so much hype about 'big data' when reality is that most organizations don't have the guts to actually do something that data clearly points out is needed.
Patrick Taylor
Patrick Taylor,
User Rank: Apprentice
11/10/2012 | 6:35:11 PM
re: 6 Lies About Big Data
Lie # 6 is the key - doing something with the analysis. Often the focus of analytic success stories is the really smart strategic decisions. But let's face it you don't make that many strategic decisions. Now think about tactical decisions - retaining a customer, what product to offer, what price to pay for supplies. There are several orders of magnitude more tactical decisions than strategic ones.

"Doing something with our analysis" at the front lines of business will drive ROI from Big Data.
The Agile Archive
The Agile Archive
When it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.