Big Data Tempest In A Teapot - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Software Platforms
Commentary
2/25/2015
12:36 PM
Doug Henschen
Doug Henschen
Commentary
Connect Directly
Google+
LinkedIn
Twitter
RSS
100%
0%

Big Data Tempest In A Teapot

Hadoop and big data community infighting won't attract enterprise adoption. It's time to raise the level of discourse.

15 Hot Skill Sets For IT Pros In 2015
15 Hot Skill Sets For IT Pros In 2015
(Click image for larger view and slideshow.)

If you followed last week's big data news, it's likely you came away dazed and confused about the state of Hadoop. And if you follow this week's headlines about Hortonworks' latest, $12.7 million quarter, you might wonder why there's such a fuss about big data.

The fact is, the big data market is still very small, and it's full of green products, discord, and factionalism. One camp, led by Pivotal and Hortonworks, last week announced the Open Data Platform, describing it as "a structured way for vendors to agree on a fully integrated and validated core distribution of Apache Hadoop." That group's claim is that the Hadoop community is fragmented. By rallying ODP members (including Hortonworks, Pivotal, IBM, SAS, and others) around ODP Core-sanctified components of Hadoop, the group hopes to focus investment and foster continuity and compatibility.

[ Want more on data analysis? Read Gartner BI Magic Quadrant 2015 Spots Market Turmoil. ]

A second camp emerged when both Cloudera and MapR declined to join ODP. Cloudera co-founder and chief strategy officer Mike Olson reasoned that the open-source Apache Foundation process has ensured a stable Hadoop trunk that every Hadoop distributor builds upon. "There’s simply no fundamental incompatibility among the core Hadoop components shipped by the various vendors," Olson says.

I have to agree with Curt Monash, who sizes up ODP as way for Hortonworks to "minimize the importance of any technical advantages Cloudera or MapR might have," and a "face-saving way" for IBM and Pivotal to let go of (or at least reduce the cost of maintaining) their Hadoop distributions. On the other hand, it's not necessarily a bad thing -- or a danger to the health of the Hadoop Community, as Olson suggests -- to see commercial vendors like Pivotal, IBM, and SAS adapting their software to run on core components of Hadoop that are available to all.

Cloudera, Hortonworks, and MapR clearly have to differentiate their offerings in order to compete and win business, but this holier-than-thou posturing and infighting is a distraction. Hortonworks' $46 million in annual revenue reported Tuesday equals that of a typical car dealership. At somewhere over $100 million in annual revenue, Cloudera isn't that much bigger.

InformationWeek's latest research, based on interviews with 374 information-management decision makers, shows that only about 4% of mainstream enterprises use Hadoop "extensively." Another 18% use it on "a limited basis," and 20% are "considering" the technology. That leaves 58% with "no current or planned use" of Hadoop.

If Hadoop is going to grow into the multi-billion dollar market that many envision, it's that 78% just starting to consider or not even looking at Hadoop that will make the difference. The likes of $86-billion-annual-revenue Microsoft and $38-billion-annual-revenue Oracle are well along in training their eyes on this market. Microsoft put an emphasis on big data analysis last week, introducing an Apache Storm service on its Azure HDInsight Cloud offering while also adding more algorithms and language options (including Python) to its Azure Machine Learning service.

Oracle is focusing on data analysis and movement, because that's where the money is -- not in merely storing data in a lake. Oracle took the wraps off its previously announced Oracle Big Data Discovery tool, and it introduced an adaptation of Oracle GoldenGate for big data streaming into HDFS, HBase, Hive, Storm, and Spark.

What Microsoft and Oracle are doing is reassuring the other 78% that they can help them if and when they're ready for big data analysis. When that day arrives, maybe that will or maybe that won't require Hadoop. Perhaps by then it will be commoditized into operating systems, as Forrester has warned.

If adoption is going to happen sooner, rather than later, what the Hadoop market needs is a constant drumbeat of user success stories. It needs use-case examples in every major industry sector. Companies large and small need to see that this is a technology they can manage.

The infighting is counterproductive and won't reassure anyone that Hadoop is ready for adoption. Tell me your technology is somehow purer and I'll yawn. Show me that a company much like mine is getting breakthrough, money-making results and I might be interested.

Attend Interop Las Vegas, the leading independent technology conference and expo series designed to inspire, inform, and connect the world's IT community. In 2015, look for all new programs, networking opportunities, and classes that will help you set your organization’s IT action plan. It happens April 27 to May 1. Register with Discount Code MPOIWK for $200 off Total Access & Conference Passes.

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
asksqn
50%
50%
asksqn,
User Rank: Ninja
2/27/2015 | 5:52:54 PM
Big Data, who?
Big Data isn't catching on because no one really knows what it actually means.  Sure, there is a lot of hype and buzzswording going on, but realistically speaking,  Big Data and what it means to industry seems to be as nebulous as IoT.
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
2/25/2015 | 4:58:20 PM
Is that teapot or teacup?
Maybe the Hadoop market is more akin to a teacup in market size. In any case, all the analysts do see huge market potential -- billions by 2020, or so they say. The Hadoop distributors talk a lot about the Linux model as their guiding light. Just FYI, Red Hat, founded in 1993 and one of the leaders in the Linux market, was a $1.5 billion-revenue-company in 2015. Clearly companies are spending much less money on open source technology -- so maybe we just won't see IBM-, Microsoft- and Oracle-scale companies in the future.
Slideshows
Strategies You Need to Make Digital Transformation Work
Joao-Pierre S. Ruth, Senior Writer,  11/25/2019
Commentary
Enterprise Guide to Data Privacy
Cathleen Gagne, Managing Editor, InformationWeek,  11/22/2019
News
Watch Out: 7 Digital Disruptions for IT Leaders
Jessica Davis, Senior Editor, Enterprise Apps,  11/18/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Getting Started With Emerging Technologies
Looking to help your enterprise IT team ease the stress of putting new/emerging technologies such as AI, machine learning and IoT to work for their organizations? There are a few ways to get off on the right foot. In this report we share some expert advice on how to approach some of these seemingly daunting tech challenges.
Slideshows
Flash Poll