How To Find Strategic Advantage From Big Data - InformationWeek
Software // Information Management
06:09 PM
Connect Directly
[Dark Reading Crash Course] Finding & Fixing Application Security Vulnerabilitie
Sep 14, 2017
Hear from a top applications security expert as he discusses key practices for scanning and securi ...Read More>>

How To Find Strategic Advantage From Big Data

Few top-level executives understand the changes necessary in data-gathering and decision-making processes well enough to make big-data migrations a real priority.

Big Data Talent War: 10 Analytics Job Trends
Big Data Talent War: 10 Analytics Job Trends
(click image for larger view and for slideshow)
Big data has become more than the hot buzzword in a technology sector still adapting to the fundamental changes created by the last wave of disruptive technologies--virtualization, cloud computing, mobile computing, and BYOD consumerization.

Whether they're prepared to deal with it or not, even mid-size companies have to face big data, not for the benefits it could offer, but because nearly all corporate data has become "big," according to reports from Ventana Research, Forrester, and other analysts.

Stereotypically big data sets--defined as any data set too large for conventional databases or analytical tools--are inflated with status reports, error messages, activity logs, and other data from the "Internet of Things," according to a seminal report from McKinsey & Co.

Big data is certainly a game changer, but it's at a stage of development in which hype overwhelms reality, according to Dan Vesset, the IDC analyst who was lead author on a March study that predicted the market for big data products and services would grow at 40% a year through 2015.

[ Do you see big data as a pain or an opportunity? Read Does IT Really Care About Big Data?. ]

The rapid growth, though it seems sudden, is actually the result of nearly 30 years' worth of development in analytic software, data warehouse techniques, and best-practice guidelines on how to manage a business based on data rather than intuition, Vesset said.

Helping the growth along is the rush among major vendors to adapt non-big-data databases and software to handle big data, as well as "more than half a billion dollars in venture capital" funding a host of startups, Vesset said.

That doesn't mean either users or vendors really understand the market.

Users are eager to build on reorganizations made to take advantage of ERP, CRM, and other products designed to put data, rather than guesswork, behind critical business decisions. Few top-level executives, however, understand the changes necessary in a company's data-gathering and decision-making processes well enough to make big-data migrations a real priority.

Still, users push their organizations toward big data, even at companies that find little value in data from the Internet of Things, according to Forrester analyst Mike Gualtieri.

Status reports from the inanimate aren't necessary for an app or a decision to qualify as being driven by big data, Gualtieri said. The only thing needed to give small data the impact of big is that the data be demonstrably causative. That is, the data being analyzed has to have a direct cause-and-effect connection with the decision being made. Big data needs to be causative; needs skilled data analysts to build and manage it; needs good-quality predictive-analytic software to boil data down to recommendations; and needs a program, platform, or set of processes that put changes in place based on decisions driven by data, Gualtieri said.

It's good for mid-size companies that all those requirements are available to them as well as to larger companies, but that availability also puts mid-size companies in the position of having to move faster on big data than they'd like, just to keep from being left behind, according to Daniel Castro, senior analyst with Information Technology and Innovation Foundation (ITIF), a Washington, D.C.-area think tank.

Among the problems mid-sized companies have adapting to big data is the kind of non-standard data it includes, primarily unstructured text from end users. That includes commentary or feedback from social networks, location data from mobile devices, forum postings, and other user-generated sources of data.

That data, almost completely excluded from BI or other assistive decision-making products until recently, has become the primary means of customer interaction for most companies, according to McKinsey analyst James Manyika.

The McKinsey study contrasts the traditional reliance on transactional information as a source of intelligence on customers with what it called "exhaust data" within big-data datasets. Exhaust data, like metadata, is content created as a by-product of other online activity. Unlike metadata, exhaust data exists on its own rather than simply as a way to provide context for records stored in traditional relational databases.

1 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Moderator
8/9/2012 | 2:05:53 PM
re: How To Find Strategic Advantage From Big Data
Kevin, good article. An enterprise-ready solution worth looking at is HPCC Systems which provides a single platform that is easy to install, manage and code. Their built-in analytics libraries for Machine Learning and integration with Pentaho for great BI capabilities make it easy for companies to understand the meanings and patterns hidden in complex information and extract the value for taking action on real-time insights. For more info visit:
User Rank: Apprentice
8/8/2012 | 8:14:29 PM
re: How To Find Strategic Advantage From Big Data
Thanks for the interesting article. Over the past year "big data" has turned from an interesting term to a catch-all phrase that seems to be used for everything including the kitchen sink. some of the people in the article seem to be saying that they are considering whether big data is important to them or not, but I think it's helpful to make a distinction between big data as a type of data and a specific big data project. The bottom line is that big data isn't a choice. I think every company should view big data as a short term critical business element. If you look at big data as data that goes beyond the limits of traditional database and BI technology in terms of how large the set can be or how diverse the data can be or how frequently that data can change, then there is no question that big data is strategic and important. Of course, not ALL big data is important and organizations are going to have to evaluate projects based on their business value. It's clear that thereGăÍs still a lot of headway to be made when it comes to tapping into big data. Telecom is a good example of how big data is important and adds value - some companies that we work with in telecom are bringing data together from 60 or more data sources and doing it in a fashion that they can make real-time decisions on potential fraud threats and ad hoc analysis of revenue data to improve their operational processes. They have been doing that for years and getting significant value (hundreds of millions of dollars worth) from those projects, but now we look at it and say it's a "big data" project. But the fundamental problem being solved there is using "data discovery" capability to solve a problem G㢠allowing users to bring together a wider array of stakeholders to explore diverse data types much more quickly, and at a much lower cost, than traditional means. In doing this, the people who need information to make strategic decisions can get much closer to relevant data. It lets people explore every possibility without going in to the analytics process with preconceived ideas about likely outcomes that can sometimes mask alternate, valuable truths hidden in the data. I talk about this a bit more, here: - John Joseph, Lavastorm Analytics
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of IT Report
In today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll