Hortonworks Releases Its (Conservative) Hadoop Platform

Spun out of Yahoo nearly a nearly a year ago, Hortonworks, the Hadoop distribution, support, training, and consulting provider, on Monday finally released its first product, Hortonworks Data Platform (HDP) 1.0.

Among the leading Hadoop distribution-and-support providers, Cloudera, is seen as the market share and momentum leader, having had four years in the market and having amassed lots of important and influential partnerships. MapR is the performance-oriented rebel, replacing flawed components like the Hadoop Distributed File Systems (HDFS) and injecting non-open-source components to provide high-availability features and performance gains many Hadoop veterans seem to welcome.

With its 1.0 release, Hortonworks is towing a conservative line, sticking with Apache's Hadoop 1.0 code line. Cloudera last week released its CDH4 distribution incorporating code from Hadoop 2.0 (specifically version .23). But Hortonworks contends that code isn't ready for production use.

"Cloudera is shipping alpha [open-source] software that hasn't been generally released," Shaun Connolly, Hortonwork's VP of corporate strategy told InformationWeek. "Our view is that it needs to bake a bit more before we unleash it on the next wave of enterprise adopters."

[ Want more on Hadoop distributions? Read Cloudera Releases Next-Generation Hadoop Platform. ]

Cloudera went ahead with 2.0 in part because it addresses the well-known, single-point-of-failure vulnerability of the HDFS NameNode. Hortonworks says it's contributing to 2.0 and will release that code "once it's ready."

For now, Hortonworks says it's solving the high-availability (HA) problem and more with virtualization technology from VMware. Thus, HDP 1.0 offers automated HDFS namenode failover and failback, as well as automated MapReduce detection and response to HDFS failover events.

"With tight integration between VMware and Hadoop 1.0, [we're delivering] a highly available system that covers not only the NameNode problem, but also the Job Tracking service and other services that can benefit from high availability," Connolly said.

The release also sticks to standards when it comes to its systems deployment, administration, and management console. Where Cloudera delivers these functions with its proprietary Cloudera Enterprise 4.0 software, Hortonwork's management and monitoring services are based on Apache Ambari, open-source software that offers dashboards that visually monitor the health of a Hadoop clusters. Components of Ambari include the open-source Nagios monitoring and Puppet configuration-management tools.

Hortonworks says another differentiator is that it is the first to use Apache HCatalog functionality for metadata services. These services enable users to define the structure and location of data within Hadoop. This feature is being exploited by Hortonworks partners including Microsoft and Teradata (specifically its Aster Data unit) to provide fast and consistent access so they can analyze data within Hadoop without actually moving it into their relational databases.

A final differentiator for Hortonworks is the inclusion of data-integration software from Talend, which moved its open-source software to Apache specifically to support big-data Hadoop environments. Talend's graphical design environment supports drag-and-drop data integration between leading data sources and data targets without requiring ETL coding.

Cloudera may have landed data warehousing leader Oracle as a partner early this year, but Hortonworks points to its hoped-for customers and partners including IBM, Microsoft, and Teradata and says it has good reason to take a conservative approach. "Early adopter technology enthusiasts will likely self support, and that's fine, but we're really targeting the pragmatists and the more conservative enterprises out there," Connolly concluded.

At the Big Data Analytics interactive InformationWeek Virtual Event, experts and solution providers will offer detailed insight into how to put big data to use in ad hoc analyses, what-if scenario planning, customer sentiment analysis, and the building of highly accurate data models to drive better predictions about fraud, risk, and customer behavior. It happens June 28.