IoT
IoT
Data Management // Big Data Analytics
News
1/12/2016
09:06 AM
50%
50%

Big Data Moves From Hype To Reality, CompTIA Finds

Organizations are moving past the hype and into actual value when it comes to big data and analytics implementation, according to a new survey by CompTIA. But challenges remain, including a skills gap and the struggle to wrangle the growing quantity of data generated.

14 Ways IoT Will Change Big Data And Business Forever
14 Ways IoT Will Change Big Data And Business Forever
(Click image for larger view and slideshow.)

Companies are moving to the next stage with big data -- past the hype and into broader adoption. But new challenges are on the horizon, such as how to master all that data. Those are a couple of several key findings in a recent Computing Technology Industry Association (CompTIA) study.

The industry organization surveyed 402 IT professionals for the report released in December and found that 72% of respondents said big data projects had exceeded their expectations, and about 75% said their businesses would be stronger if they could harness all of their data.

Companies "are trying to consolidate data and understand the structure of their data," said Seth Robinson, senior director for technical analysis at CompTIA, in an interview with InformationWeek. They are finding this out the hard way, because lax data policies in the past allowed departments and work units to "do their own thing" in IT, resulting in data silos. "Companies have to look at data differently now."

Data silos are a problem for 42% of IT professionals responding to the survey, up from 29% two years ago.

More Data Than Ever

Plus, more data is being generated than ever before. The CompTIA report cites another study by Cisco that forecast 2016's IP traffic passing the 1 zettabyte threshold. That is 1 billion TB of data. That number will double again in three years. Helping to drive the data glut, the price of storage has dropped from $1,000 per GB in 1995 to a few pennies today. So, squirreling away 100TB of data is not going to be a big deal, especially now that companies can rent storage in the cloud to suit needs.

[Algorithms are changing the world. But where did they start? Read Algorithms: Turning Data Into Products.]

How Organizations Use Data

Yet 63% of survey respondents said they rely on data for day-to-day operations, 61% said that they are more sensitive to data privacy issues, and 59% said that they want to use data to better understand their customers.

Respondents are evenly split when asked how and where they seek improvement in their operations. For example, 40% said they are doing well with real-time analysis, while 42% seek improvement. A full 39% of respondents want to be able to search all of their data across departmental lines, while 37% say they want to improve their ability to do so.

(Image: matdesign24/iStockphoto)

(Image: matdesign24/iStockphoto)

In 2015, 51% of all respondents said they were undertaking big data projects, while another 36% said they were planning to do so. This is up from 2013, when 42% of respondents said they were undertaking big data projects, and 46% said they were planning them.

Despite the desire to reap the fullest benefit from data, companies undertaking big data projects are not suffering setbacks. "They are not running into headaches or hidden costs," Robinson said. Companies may be meeting modest goals. "The metric may not be that obvious," he said. "Coming up with the right metric is part of being data savvy."

Skills Gap

Still, there is a skills gap that needs to be addressed. A total of 49% of respondents said they have sufficient data skills in-house to take care of big data projects. But 40% admit they are "moderately deficient." The biggest skills gaps are real-time analytics (reported by 42%), relational databases (41%), data security (36%), predictive analysis (35%), distributed storage (33%), and data mining (30%).

"Part of this is technical," Robinson observed. The tools needed to handle big data are new, so there may be insufficient experience to make full use of their potential. There is also an informal skill set needed as well, since data analysts need to "speak" business as well as understand the technology, he added.

Enterprises may turn to outsourcing to help fill this knowledge gap, but only if they can find the right skills there. "We heard companies saying when they go outside, they can't find the knowledge or the skills," Robinson said.

The report noted that third parties are still concentrating their expertise on component services and not end-to-end solutions.

Still, component expertise can become a building block toward providing a big data fix. Data backup, for instance, could lead to "having a conversation about the data," Robinson said, while installing a firewall can be a step toward greater data security.

"People have gotten past thinking it (big data) is hype," Robinson said. Companies are building on what they have learned, which in turn can lead them to more comprehensive big data solutions, he added. "We'll see that play out over this year."

**Elite 100 2016: DEADLINE EXTENDED TO JAN. 15, 2016** There's still time to be a part of the prestigious InformationWeek Elite 100! Submit your company's application by Jan. 15, 2016. You'll find instructions and a submission form here: InformationWeek's Elite 100 2016.

William Terdoslavich is an experienced writer with a working understanding of business, information technology, airlines, politics, government, and history, having worked at Mobile Computing & Communications, Computer Reseller News, Tour and Travel News, and Computer Systems ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Broadway0474
50%
50%
Broadway0474,
User Rank: Ninja
1/15/2016 | 11:58:37 PM
Re: Promising outlook
Kevin, what does "exceed expectations" mean? What were their expectations? Were they very low to begin with? It could mean a lot that three out of four respondents were pleasantly surprised by results, unless it doesn't. We need more info to get excited, right?
HM
50%
50%
HM,
User Rank: Moderator
1/15/2016 | 9:48:26 AM
Nice perspective!
William, very nice perspective on Big Data. We are seeing an increase in businesses seeking specialized skills to help address challenges that arose with the era of big data. The open source HPCC Systems platform from LexisNexis helps to fill this gap by allowing data analysts themselves to own the complete data life cycle. Designed by data scientists, the programming language called ECL is declarative and expresses data algorithms across the entire HPCC platform. Their built-in analytics libraries for Machine Learning and BI integration provide a complete integrated solution from data ingestion and data processing to data delivery. HPCC Systems provides proven solutions to handle what are now called Big Data problems, and have been doing so for more than a decade. Their free online introductory courses allow for students, academia and other developers to quickly get started. For more info visit: hpccsystems.com
pfretty494
50%
50%
pfretty494,
User Rank: Moderator
1/13/2016 | 10:54:03 AM
Maturity is steady
Big data as a whole has grown as a result of the hype, which is crucial considering the role analytics will play as we move towards an IoT environment. There is still a strong need for the queries to start coming from the business rather than IT and data scientists. This is going to take time to build the data culture. Peter Fretty, IDG blogger for SAS
Ariella
50%
50%
Ariella,
User Rank: Author
1/12/2016 | 5:04:41 PM
Re: Promising outlook
We do have to bear in mind that the results here are based on the responses of only 402 people. It's possible that's not a large enough sample to really gauge big data trends for businesses in general. 
Ariella
50%
50%
Ariella,
User Rank: Author
1/12/2016 | 5:02:51 PM
Re: It's already done.
@Ilya If you say it is, in fact, structured and then quote IBM saying it can process unstructured data, are you saying that IBM misrepresents itself?
KevinPetrieTech
50%
50%
KevinPetrieTech,
User Rank: Apprentice
1/12/2016 | 2:17:52 PM
Promising outlook
Very useful findings here.  The data point that 72% of respondents said big data projects had exceeded their expectations implies we might be futher along the maturity curve than is commonly thought.  Using Gartner's model, this suggests we are moving past the trough of disillusionment toward the plateau of productivity.

@KevinPetrieTech

www.attunity.com
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Listen Now InformationWeek Live for the Week of September 25, 2016
Join us for a roundup of the top stories on InformationWeek.com for the week of September 25, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.