Big Data Success Remains Elusive: Study - InformationWeek
IoT
IoT
Data Management // Big Data Analytics
News
2/2/2015
09:40 AM
Connect Directly
Google+
RSS
E-Mail
100%
0%
RELATED EVENTS
[Dark Reading Crash Course] Finding & Fixing Application Security Vulnerabilitie
Sep 14, 2017
Hear from a top applications security expert as he discusses key practices for scanning and securi ...Read More>>

Big Data Success Remains Elusive: Study

Just over a quarter of organizations say their big data initiatives are a success, according to a recent Capgemini study. So why are three of four unsuccessful?

9 CIO Tech Priorities For 2015
9 CIO Tech Priorities For 2015
(Click image for larger view and slideshow.)

Nearly eight out of ten organizations have big data projects underway, but only 27% describe their efforts as "successful," and a scant 8% as "very successful." But despite this dim view of their data-driven efforts thus far, 60% of executives surveyed recently by consulting firm Capgemini say big data will "disrupt their industry" within three years.

At first glance, the juxtaposition of failed implementations and high expectations may seem odd. Why the enthusiasm for a technology that doesn't appear to be paying off? According to Jeff Hunter, Capgemini's vice president of North American business information management, the high failure rate of big data initiatives isn't all that surprising, at least not initially.

"If we look at it in the analogy of other technologies that have come along the way -- a website, then digital presence, digital ecommerce, digital store, payments, and so forth -- we saw the same type of errors in the beginning of those technology trends," he said in a phone interview with InformationWeek.

The Capgemini survey of big data executives in November 2014 included 226 respondents in Europe, North America, and APAC (Asia-Pacific). It spanned multiple industries, including energy and utilities, financial services, manufacturing, pharmaceuticals, and retail.

A common situation with technology initiatives is that executives are often anxious to try something new and hyped -- in this case, big data -- partly in fear of falling behind their competitors. As a result, there's no clear big data mandate from the C-Suite, no well-defined strategy to improve, modify, or invent.

(Image: DARPA)
(Image: DARPA)

These ill-defined objectives are the primary cause of the majority of big data failures. "Generally, it's a disconnect between the output and a clearly defined business driver or goal," said Hunter. "And along the way, people get engulfed in the technology."

In some cases, organizations attempt to use their existing data management systems to process big data streams, often with poor results.

"Legacy systems that generally have been used to great efficiency for enterprise data management and content management, sometimes aren’t suited to these new data sources," Hunter said.

These sources may include social media streams, log data, and sensor data from the emerging Internet of Things to evaluate customers, transactions, and user sentiment. But this approach usually doesn’t go well, resulting in what he calls a "fumbling of the legacy systems."

Another problem is scattered silos of data. The Capgemini report states:

Seventy-nine percent of organizations have not fully integrated their data sources across the organization. This means decision-makers lack a unified view of data, which prevents them from taking accurate and timely decisions.

So how can organizations achieve greater success with their big data initiatives? The key is a well-defined organizational structure, a systematic implementation plan, and strong leadership, Hunter said.

[Does everyone need to be a data analyst? Read Data Analyst: Does Everybody Need To Be One?]

Big data initiatives are rarely "division-centric," but rather cut across multiple departments, the report states. Success rates for organizations with an analytics business unit are nearly 2.5 times higher than those with "ad-hoc, isolated teams," the survey found.

"The firms that we see succeed [have] centralized the concept of consumption of big data technology, leverage of data science, and application of analytics," said Hunter. "They’re the ones we see moving faster."

Attend Interop Las Vegas, the leading independent technology conference and expo series designed to inspire, inform, and connect the world's IT community. In 2015, look for all new programs, networking opportunities, and classes that will help you set your organization’s IT action plan. It happens April 27 to May 1. Register with Discount Code MPOIWK for $200 off Total Access & Conference Passes.

Jeff Bertolucci is a technology journalist in Los Angeles who writes mostly for Kiplinger's Personal Finance, The Saturday Evening Post, and InformationWeek. View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
pfretty
100%
0%
pfretty,
User Rank: Ninja
2/4/2015 | 11:50:05 AM
Slowly maturing
There is a good reason why success is eluding organizations.  There needs to be a little more devotion to building the data maturity level.  According to a recent IDG survey, organizations do not rate themselves as highly effective in any of the critical big data tasks. For instance, only 11 percent say their organizations are extremely capable at knowing which questions to ask.  Likewise, only 5 percent see their organizations as effective at disseminating insights across the organization. When you couple this with businesses trying for the big wins immediately, you are bound to see disappointment. Bottomline, businesses need to start small, build confidence and ability with experience. 

 

Peter Fretty, IDG blogger working on behalf of SAS
_cazena
100%
0%
_cazena,
User Rank: Apprentice
2/3/2015 | 10:51:00 AM
an accurate summation of the state of big data in the enterprise
Great stuff! The article and survey are consistent with what we see at Cazena. Enterprise business leaders see the promise of big data to transform the speed and quality of their decision-making, but struggle with the complexity of the new technologies and other practical matters that so far keep big data mostly on the fringes of true production-level success. As is pointed out, this is a natural phase of growth that we'll get through if we keep our eye on the prize!
Li Tan
50%
50%
Li Tan,
User Rank: Ninja
2/3/2015 | 2:30:22 AM
Re: Data science
I do agree with you. Big data is not something that can be handled well by pure technician. You need to understand the real business need behind to dig out valuable information from existing data mine.
danielcawrey
50%
50%
danielcawrey,
User Rank: Ninja
2/2/2015 | 9:33:31 PM
Data science
I think that the data science aspect of this is really important – companies that use an actual data scientist who knows what to do with data is really important I think. 

Big data is not just another facet of technology. You need the right people looking at the data in order to gain insight into what it all really means. 
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of IT Report
In today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll