Top 5 Big Data Trends Of 2014 - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Software Platforms
Commentary
12/8/2014
11:30 AM
Doug Henschen
Doug Henschen
Commentary
Connect Directly
Google+
LinkedIn
Twitter
RSS
100%
0%

Top 5 Big Data Trends Of 2014

As companies move beyond bleeding-edge experiments into production deployments, these trends point to real-world progress in big data analysis.

InformationWeek Chiefs Of The Year: Where Are They Now?
InformationWeek Chiefs Of The Year: Where Are They Now?
(Click image for larger view and slideshow.)

The era of big data analysis is here to stay. Take your pick of 2014 proof points.

Tech watchers might cite the more than $200 million in venture capital raised by the top three NoSQL database vendors, or the $1 billion raised by the top-three Hadoop software distributors. Many took note of the recent declaration by Forrester Research that "Hadoop is no longer optional" for large enterprises, thanks to compelling "Hadooponomics" that make it a must for high-scale storage and data processing.

InformationWeek is more impressed by the testimonials of companies that are getting real value out of big data platforms and analysis techniques. Pfizer and Merck, for example, are developing more effective and affordable drugs thanks to big data techniques that are leading to more targeted treatments and more productive manufacturing processes. GE and others are demonstrating improvements in industrial equipment performance, uptime, and safety thanks to Internet of things-style applications.

[Want more on the top IT achiever of 2014? Read IT Chief Of The Year: Bank Of America's Cathy Bessant.]

And then there are the pioneers like The Weather Company and Facebook that say they just couldn't run their data-driven businesses without new platforms, even if they still have a place for more conventional tools like relational databases.

Here are five trends witnessed over the last year that point to progress in big data analysis:

1. SQL meets Hadoop
Hadoop is here to stay, so every data management vendor worth its salt must have a SQL-on-Hadoop or SQL-access-to-Hadoop option. Here are five of our most-read stories in the SQL-meets-Hadoop vein:

Just remember that SQL is not designed to find correlations among variably structured data sets. Nor does it support machine learning, many advanced analytics techniques, or other approaches often associated with big data analysis. If SQL solved everything, we wouldn't need new platforms.

2. Platforms mature
Every other week in 2014, or so it seems, Hadoop software distributors and NoSQL database vendors announced new management consoles, security systems, data management capabilities, search engines, or high-availability features. Here's a sampling of what we're talking about:

These and other big data vendors are trying to reassure enterprise IT types that these products are secure and reliable as 30-year-old database management systems. Let's just say that more than a few grizzled IT veterans are still used to working with favored and familiar tools and still need some convincing.

3. Educational options proliferate
Nature abhors a vacuum, so into the void of data science and big-data platform knowledge and expertise have rushed vendors,

Next Page

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Previous
1 of 2
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Page 1 / 2   >   >>
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
12/15/2014 | 11:31:57 AM
Re: SQL on Hadoop
HAWQ was introduced in 2013, not 2014, so it was ahead of the 2014 rush of followers. I haven't seen any difinitive benchmarks about HAWQ versus any other SQL-on-Hadoop option ("Stinger" or otherwise). I've also seen no evidence of big HAWQ adoption. If it's so great, why haven't we seen lots of HAWQ users sharing their stories? I'm open to writing about them if Pivotal can make customers available. Ditto Presto, which I hear about mainly from Presto advocates (not practitioners).
pfretty
50%
50%
pfretty,
User Rank: Ninja
12/15/2014 | 11:23:23 AM
Love the list
I think you are spot on with your trend identification.  I would like to see a little more focus on developing, implementing and nurturing strategies. According to a recent IDG survey, data strategy development is still lagging considerably, which could prove discouraging to organizations not realizing the ROI as quickly as they hope. 


Peter Fretty, j.mp/pfrettysa
tspann085
50%
50%
tspann085,
User Rank: Apprentice
12/15/2014 | 10:08:19 AM
SQL on Hadoop
You cannot forget about the fastest SQL engine on Hadoop, Pivotal HAWQ which is much faster than Stringer or Presto.    It also has full SQL support and can run all TPC-DS 111 queries.
HM
50%
50%
HM,
User Rank: Moderator
12/9/2014 | 3:43:34 PM
Big Data Alternative
Doug, interesting trends on Big Data. Hadoop alternatives such as HPCC Systems provides proven solutions to handle what are now called Big Data problems, and have been doing so for more than a decade. More info at http://hpccsystems.com.

 
gkumar_splice
50%
50%
gkumar_splice,
User Rank: Apprentice
12/9/2014 | 3:09:54 PM
Re: A remaining challenge?
I agree with Charlie's comment, and I believe in addition to real-time, we need to also ensure that operational applications, which require high concurrency and consistency requirements, can leverage Hadoop's scalability and fault-tolerant capabilities. Splice Machine's RDBMS offers organizations these capabilities on top of the Hadoop stack - real-time updates, support for applications with 100s of users and high concurrency, and those which require consistency guarantees. 
Laurianne
50%
50%
Laurianne,
User Rank: Author
12/9/2014 | 9:34:10 AM
2015
Will 2015 be the year more companies build/train their own data analysis gurus? I hope we continue to see more IT people who may not have formal analytics training establish their expertise via project work.
yamtssfa
50%
50%
yamtssfa,
User Rank: Apprentice
12/9/2014 | 6:43:29 AM
Big data - but from where?
I think it's also important to mention the trends of data acquisition, either private data (the data service companies collect about us), or aggregated data (Gnip/Datasift), or publicly available data (Webhose.io/Moreover)
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
12/8/2014 | 7:04:59 PM
Re: A remaining challenge?
It would be fair to say that streaming analysis remains a challenge, but there are multiple initiatives trying to address that problem. In the big data arena examples include Spark, Storm, Amazon Kinesis. NoSQL and NewSQL vendors and options are also sprouting analytical features, but I'd say this trend hasn't really hit the production mainstream this year.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
12/8/2014 | 6:56:42 PM
A remaining challenge?
Would it be fair to say that a remaining challenge for big data practitioners is getting their systems to produce results closer to real time, as opposed to batch timje? So much could be done if we could instantly extract the nuggets we need from the big data generated on active Web sites or in ecommerce applications. Getting useful data to respond to a site visitor, or to recognize a significant event amidst a stream of them -- that's a big challenge.
Stratustician
50%
50%
Stratustician,
User Rank: Ninja
12/8/2014 | 3:33:04 PM
is 2015 the year of big data?
Nice to see some actual use cases for Hadoop start to come to light.  A year ago folks were still trying to figure out it seems whether Big Data analytics was more than hype.  I guess here we have some great proof points as to why it's going to be critical for many organizations moving forward.  Even nicer to see is that the right educational tools are in place to support these new areas of IT.
Page 1 / 2   >   >>
Slideshows
7 Technologies You Need to Know for Artificial Intelligence
Jessica Davis, Senior Editor, Enterprise Apps,  7/1/2019
Commentary
A Practical Guide to DevOps: It's Not that Scary
Cathleen Gagne, Managing Editor, InformationWeek,  7/5/2019
Commentary
Diversity in IT: The Business and Moral Reasons
James M. Connolly, Editorial Director, InformationWeek and Network Computing,  6/20/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
A New World of IT Management in 2019
This IT Trend Report highlights how several years of developments in technology and business strategies have led to a subsequent wave of changes in the role of an IT organization, how CIOs and other IT leaders approach management, in addition to the jobs of many IT professionals up and down the org chart.
Slideshows
Flash Poll