Cray Brings Hadoop To High-Performance Computing

5 Big Wishes For Big Data Deployments

Cray announced Thursday that it's putting Hadoop software on its supercomputers, bringing an "enterprise-strength approach" to "high-value Hadoop applications."

Cray is joining the Hadoop crowd by pairing Intel's Hadoop distribution with the Cray CS300 series of high-performance cluster supercomputers. The resulting system, which will be available by the end of the month, will handle complex algorithms that go beyond basic storage and reporting that are "pushing the limits of [existing] architectures," said Bill Blake, senior VP and CTO of Cray, in a statement.

"Organizations can now focus on scaling their use of platform-independent Hadoop software, while gaining the benefits of important underlying architectural advantages from Cray and Intel," Blake said.

[ Want more on Cray's CS300 product line? Read Cray Builds A Budget-Minded Supercomputer. ]

Cray listed Intel Hadoop distribution advantages, including "greater security, improved real-time handling of data, and improved performance throughout the storage hierarchy," with the "greater than" and "improved" comparisons presumably being to unoptimized Apache Hadoop software and distributions thereof from the likes of Cloudera, Hortonworks, MapR and IBM.

One of Hadoop's big cost advantages stems from the fact that it's generally deployed on clusters of commodity hardware, with the sheer number of servers and cores trumping the processing power of high-end boxes.

Cray also relies on "standard" Intel servers, but CS300 clusters are quite a bit more exotic than ordinary racks, with some even featuring liquid cooling. What's more, Cray's software stack is expressly designed for high-performance computing (HPC) and high availability. Cray's least expensive CS300starts at $500,000.

"The convergence of data-intensive HPC and high-end commercial analytics is forming a new big data market IDC calls High Performance Data Analysis," said IDC analyst Steve Conway in a statement issued by Cray. "Pairing the Cray CS300 systems with Intel's Hadoop distribution creates a solution with the potential to tackle big data problems that would frustrate most clusters."

Cray did not cite examples of Hadoop-on-Cray beta customers or specific industry scenarios calling for the ultimate in Hadoop performance. But it's evident that Cray is already entrenched in big data problems. Cray has Sonexion storage systems offering fast-data and big-data-movement capabilities. It also provides the HPC platform behind YarcData's Urika appliance for graph analytics.

These influential and accomplished government IT leaders are finding ways to be cost efficient and still innovate. Also in the new, all-digital Government CIO 25 special issue of InformationWeek Government: Video surveillance isn't just about networked cameras anymore. New technology provides analytics, automation, facial recognition, real-time alerts and situational-awareness capabilities. (Free registration required.)