Hortonworks, Pivotal Expand Open Source Big Data Alliance

12 Inspiring Women In Data Science, Big Data

Hadoop distribution is really concentrated among three major vendors who focus on big data -- Cloudera, Hortonworks, and MapR.

IBM and Pivotal have offered distributions of Hadoop, too, but unlike the other three, they offer other technologies, too.

But this week the Hadoop distributor field has narrowed as Pivotal and Hortonworks expanded an existing partnership that calls for each of them to standardize on each other's core offerings.

That means Pivotal will drop its own distribution of Hadoop and instead standardize on Pivotal HDP, described by the companies as a support subscription software that is "100% identical" to the Hortonworks Data Platform (HDP).

HDP is Hortonworks' stack of open source big data technologies that includes Hadoop.

Hortonworks will do the same for an open source database technology driven by Pivotal. Hortonworks will offer professional customer support and professional implementation services for Hortonworks HDB, a distribution of the database technology described as Hadoop-native SQL powered by Apache HAWQ, which comes out of Pivotal.

"It's a good move for both Pivotal and Hortonworks," Forrester Research Principal Analyst Mike Gualtieri told InformationWeek in an interview. "Pivotal can now focus their efforts on their HAWQ database. It's a good database and SQL engine. And when they go to customers to sell a solution they don't have to answer the questions about why [there are] so many Hadoop distributions. They are going with Hortonworks, one of the leaders."

Hortonworks also gains from this expanded partnership.

"They are getting the HAWQ database and filling a real gap in SQL technology in their platform," Gualtieri said. Hortonworks had been relying on Apache Hive, Gualtieri said, and that technology hadn't been able to catch up with Impala, the one championed by rival Cloudera.

ODPi

Hortonworks and Pivotal forged their original alliance in February 2015, soon after both companies became founding members of the Open Data Platform Initiative. The ODPI, also founded in early 2015, describes itself as a "nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification called ODPi Core."

Learn to integrate the cloud into legacy systems and new initiatives. Attend the Cloud Connect Track at Interop Las Vegas, May 2-6. Register now!

IBM is also a member of ODPi, but the two other Hadoop distributors, Cloudera and MapR are not. MapR has said it had been invited to join, but believes that ODPI mostly benefits Hortonworks marketing.

Pivotal and Hortonworks both pledged their commitment to open source and the tenets behind ODPI in the statement announcing the expanded partnership.

"This collaboration brings to life our shared belief in an open source approach and the pervasive adoption of enterprise-ready modern data platforms," said Leo Spiegel, senior vice president for corporate development and strategy at Pivotal, in the statement. "Together, Pivotal and Hortonworks have a unified approach via the ODPi Core which removes key barriers to the widespread enterprise adoption of Hadoop and modern big data applications."

Hortonworks shared a similar perspective.

"Hortonworks and Pivotal are aligned in the belief that the best way to deliver infrastructure platform technology today is to eliminate fragmentation and do it completely in open source," said Herb Cunitz, president at Hortonworks, in the statement.

The companies said that Hortonworks HDB will be available as an add-on support subscription to HDP in the second quarter of 2016.