Informatica Deal Promises Big Data Impact

EMC Greenplum, Netezza, Teradata: Cloudera has partnerships with these and other data warehousing luminaries. But a deal announced by Informatica on Monday just may outclass all those alliances.

The fruit of the new partnership will be a co-developed connector between the Informatica data integration platform and Cloudera's distribution of open source Apache Hadoop. Expected to be released early next year, the connector will give Informatica an on ramp to large-scale data-analysis projects in carried out in Hadoop.

For Cloudera, the connector will ease adoption of Hadoop for the tens of thousands of developers who are already familiar with Informatica's integration software.

"The relationship with Informatica is obviously a big deal for us," said Mike Olson, CEO of Cloudera. "There are 3,000 trained users of Informatica's software inside Accenture alone, and that's just one integrator." Informatica has more than 4,200 customer firms in total.

Built on low-cost commodity hardware, Hadoop deployments can store big data at very low cost. Support for MapReduce parallel data processing operations makes Hadoop ideal for studying inconsistent data or non-relational data such as text (think e-mail messages or social media comments) and images. Hadoop also shines in supporting complex analyses involving mixed data types.

Cloudera CEO Olson

"JP Morgan Chase looks for escalated risk in its mortgage portfolio by examining other customer behavior using Hadoop clusters," Olson said.

If somebody stops getting direct deposits in their checking account and starts buying gas on a credit card, for example, you might surmise that they've lost their job and you might want to rescore the risk of that loan. "MapReduce provides the plumbing to do that sort of analysis," Olson explained.

Informatica plans to support Hadoop as part of what it describes as its hybrid platform, meaning users can manage data integration needs for Hadoop in the same environment they use for integration with more conventional databases, data warehouse environments and content stores.

The connector and hybrid support will obviously make it easier for Informatica veterans to work with Hadoop. But don't count on an opening of floodgates. Best estimates put the number of Hadoop deployments in the thousands. Cloudera currently has fewer than 100 customers subscribing to enterprise support for the vendor's Hadoop distribution, according to a source at the company.

Apache Hadoop is used by big-data leaders including Yahoo!, Facebook and eBay. Notable Cloudera service customers include ComScore, LinkedIn, and Rapleaf. Cloudera's fast-growing partner list and the tailwind provided by the Informatica deal will surely fuel more deployments.