Top 5 Big Data Trends Of 2014 - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Software Platforms
Commentary
12/8/2014
11:30 AM
Doug Henschen
Doug Henschen
Commentary
Connect Directly
Google+
LinkedIn
Twitter
RSS
100%
0%

Top 5 Big Data Trends Of 2014

As companies move beyond bleeding-edge experiments into production deployments, these trends point to real-world progress in big data analysis.

headhunters, universities, MOOC providers, and others offering various forms of training to big-data-analysis wannabes. Here a sampling of related developments in 2014:

With education and training opportunities flourishing, there will no doubt be waves of new talent available by the time we meet the graduating classes of 2015, 2016, and 2017.

4. Cloud options multiply
Hadoop, NoSQL databases, analytics tools and platforms: You name the technology and businesses are likely to start experiments in the cloud. And many will stay there, having no interest in deploying servers and administering software on premises. That's certainly true of small and midsize practitioners we've met. Vendors are responding to the demand. Here are few of the notable cloud-oriented big data announcements made in 2014:

5. Focus turns to analysis
Data platforms will inevitably be commoditized. The real value in data is delivered through analysis, not just putting it all in a lake or on a data hub. Apache Spark offers a compelling promise: Machine learning, SQL, R-based analytics, graph network analysis, and streaming analysis all on one system. Support soared in 2014. Here's a sampling of related coverage this year:

Spark detractors (perhaps threatened) are whispering that Spark is too green or that niche alternatives (like Apache Storm for streaming analysis) might be better. Spark developer Databricks is responding with system tweaks and benchmark tests said to prove scalable performance.

Rest assured the real competitive battle in big data will be to lead in providing tools and capabilities for data analysis. Multiple commercial vendors (including Actian, Pivotal, and Teradata) seem to be aping the multi-analysis-engine platform strategy, and Cloudera's recent acquisition of DataPad, which offered a Python-based data analysis library, showed it's headed deeper into analytics.

Fasten your seatbelts -- it's going to be a competitive, and interesting, 2015.

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Previous
2 of 2
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Page 1 / 2   >   >>
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
12/15/2014 | 11:31:57 AM
Re: SQL on Hadoop
HAWQ was introduced in 2013, not 2014, so it was ahead of the 2014 rush of followers. I haven't seen any difinitive benchmarks about HAWQ versus any other SQL-on-Hadoop option ("Stinger" or otherwise). I've also seen no evidence of big HAWQ adoption. If it's so great, why haven't we seen lots of HAWQ users sharing their stories? I'm open to writing about them if Pivotal can make customers available. Ditto Presto, which I hear about mainly from Presto advocates (not practitioners).
pfretty
50%
50%
pfretty,
User Rank: Ninja
12/15/2014 | 11:23:23 AM
Love the list
I think you are spot on with your trend identification.  I would like to see a little more focus on developing, implementing and nurturing strategies. According to a recent IDG survey, data strategy development is still lagging considerably, which could prove discouraging to organizations not realizing the ROI as quickly as they hope. 


Peter Fretty, j.mp/pfrettysa
tspann085
50%
50%
tspann085,
User Rank: Apprentice
12/15/2014 | 10:08:19 AM
SQL on Hadoop
You cannot forget about the fastest SQL engine on Hadoop, Pivotal HAWQ which is much faster than Stringer or Presto.    It also has full SQL support and can run all TPC-DS 111 queries.
HM
50%
50%
HM,
User Rank: Moderator
12/9/2014 | 3:43:34 PM
Big Data Alternative
Doug, interesting trends on Big Data. Hadoop alternatives such as HPCC Systems provides proven solutions to handle what are now called Big Data problems, and have been doing so for more than a decade. More info at http://hpccsystems.com.

 
gkumar_splice
50%
50%
gkumar_splice,
User Rank: Apprentice
12/9/2014 | 3:09:54 PM
Re: A remaining challenge?
I agree with Charlie's comment, and I believe in addition to real-time, we need to also ensure that operational applications, which require high concurrency and consistency requirements, can leverage Hadoop's scalability and fault-tolerant capabilities. Splice Machine's RDBMS offers organizations these capabilities on top of the Hadoop stack - real-time updates, support for applications with 100s of users and high concurrency, and those which require consistency guarantees. 
Laurianne
50%
50%
Laurianne,
User Rank: Author
12/9/2014 | 9:34:10 AM
2015
Will 2015 be the year more companies build/train their own data analysis gurus? I hope we continue to see more IT people who may not have formal analytics training establish their expertise via project work.
yamtssfa
50%
50%
yamtssfa,
User Rank: Apprentice
12/9/2014 | 6:43:29 AM
Big data - but from where?
I think it's also important to mention the trends of data acquisition, either private data (the data service companies collect about us), or aggregated data (Gnip/Datasift), or publicly available data (Webhose.io/Moreover)
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
12/8/2014 | 7:04:59 PM
Re: A remaining challenge?
It would be fair to say that streaming analysis remains a challenge, but there are multiple initiatives trying to address that problem. In the big data arena examples include Spark, Storm, Amazon Kinesis. NoSQL and NewSQL vendors and options are also sprouting analytical features, but I'd say this trend hasn't really hit the production mainstream this year.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
12/8/2014 | 6:56:42 PM
A remaining challenge?
Would it be fair to say that a remaining challenge for big data practitioners is getting their systems to produce results closer to real time, as opposed to batch timje? So much could be done if we could instantly extract the nuggets we need from the big data generated on active Web sites or in ecommerce applications. Getting useful data to respond to a site visitor, or to recognize a significant event amidst a stream of them -- that's a big challenge.
Stratustician
50%
50%
Stratustician,
User Rank: Ninja
12/8/2014 | 3:33:04 PM
is 2015 the year of big data?
Nice to see some actual use cases for Hadoop start to come to light.  A year ago folks were still trying to figure out it seems whether Big Data analytics was more than hype.  I guess here we have some great proof points as to why it's going to be critical for many organizations moving forward.  Even nicer to see is that the right educational tools are in place to support these new areas of IT.
Page 1 / 2   >   >>
Slideshows
15 Best Places to Work in Technology 2020
Cynthia Harvey, Freelance Journalist, InformationWeek,  4/8/2020
Commentary
Future IT Teams Will Include More Non-Traditional Members
Lisa Morgan, Freelance Writer,  4/1/2020
News
COVID-19: Using Data to Map Infections, Hospital Beds, and More
Jessica Davis, Senior Editor, Enterprise Apps,  3/25/2020
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
IT Careers: Tech Drives Constant Change
Advances in information technology and management concepts mean that IT professionals must update their skill sets, even their career goals on an almost yearly basis. In this IT Trend Report, experts share advice on how IT pros can keep up with this every-changing job market. Read it today!
Slideshows
Flash Poll