Teradata Builds Big Data Pipeline To Hadoop

Teradata has teamed up with Hortonworks, the Hadoop spin-off that came out of Yahoo, to build a data pipeline and cooperative data exchange tools between the Hortonworks Data Platform and the Teradata Database and Aster analytical tools.

Teradata is behind some of its major competitors in forging a link to Hadoop but spokesmen for the two companies indicated this alliance is more than one of short-term convenience. Teradata has come to view Hadoop as "a data refining platform" that is ideal for preparing data that will be fed into downstream data analysis tools," such as Teradata's Aster, said Tasso Argyros, VP of product management at Teradata, in an interview. Aster combines SQL with NoSQL functionality to act as a "tool to discover insights hidden deep in the data," he said.

In addition to creating a data pipeline between their respective systems, Teradata and Hortonworks will pair up to offer a reference guide on how to make use of Hadoop plus SQL systems in joint operations. Shaun Connolly, VP of corporate strategy at Hortonworks, said in an interview that there was considerable confusion in the marketplace over how to use Hadoop.

[ Want to learn more about the variety of Hadoop systems now in the marketplace? See Hadoop Players Question Forrester's Take On Leaders. ]

"We will guide customers on the right way to use these complementary technologies to create new business value," he said. One of the biggest changes that's come about with the introduction of Hadoop has been the ability of it and other big data systems to refine unstructured data and feed it into downstream systems. Knowing when to use Hadoop in such a fashion is still under debate. The Teradata-Hortonworks reference guide will show use cases and "concrete guidance" on how to use the tools to attack big data problems, said Argyros.

"There are new problems that didn't exist five years ago. Enterprises and customers are not clear on which tool to use with which use case," Connolly said.

In addition, Hortonworks and Teradata will work together on joint marketing initiatives.

Commercial and open source implementations of Hadoop have proliferated in the marketplace, based on accounts of how useful it has been to Yahoo and Rackspace, among others. Yahoo analyzes its Web crawl data with Hadoop; Rackspace managed-services customer Mailtrust uses Hadoop to analyze 150 GB of mail-server log data each day for its customers.

Both commercial and open source implementations of Hadoop combine Hadoop's core distributed file system and MapReduce, a scale-out data sorting mechanism. Together, they can handle masses of data beyond the capacities of commercial relational systems.

Teradata is not the first major vendor to opt to work with Hortonworks, which includes a large contingent of former Yahoo developers who had produced a respected version of Hadoop for production use. Hortonworks spun out of Yahoo last June. Microsoft recently announced the next version of its database system, SQL Server 2012, will include Hadoop, with help from Hortonworks. It also offers a Hadoop service on its Windows Azure cloud.

Oracle adopted Cloudera's ease of use frontend and management tools for its implementation of Hadoop. IBM offers a Hadoop platform, BigInsights, as well as Hadoop-integrated InfoSphere Stream, a complex event processing system.

As enterprises ramp up cloud adoption, service-level agreements play a major role in ensuring quality enterprise application performance. Follow our four-step process to ensure providers live up to their end of the deal. It's all in our Cloud SLA report. (Free registration required.)