Hadoop Security: Some Enterprises Miss Risks

Hadoop has plenty of advocates, most of whom praise the open-source framework's speedy data processing and analytics capabilities for organizations that manage huge volumes of data. But many enterprises don't understand how to secure Hadoop, some Hadoop community members say.

Officials of Red Lambda, an Orlando, Florida-based tech company that sells MetaGrid, a large-scale Big Data analytics, automation, and security platform, believe that Hadoop's security weaknesses are being overlooked or even ignored by many enterprises that use the distributed computing system.

Hadoop "really isn't designed to be a secure processing environment, which is a little scary considering how many people are trying to use it," said Robert Bird, president, CTO, and co-founder of Red Lambda, in a phone interview with InformationWeek.

[ What else don't you know about Hadoop? See 7 Deadly Sins Of Big Data Users. ]

One of Hadoop's key weaknesses is that it lacks a system to ensure fair sharing of its resources, Bird believes. "For example, you might have an administrator of the system that's running tasks on an Hadoop cluster," said Bird. "And another user could come along, start up their own task, and completely wipe out the tasks of the original admin."

Hadoop lacks controls or protections to keep this from happening, said Bird, adding that "if you're going to have a large, super-powerful distributed computing resource, you'd think that more than one person at a time should be able to use it."

"We see Hadoop being used to solve one problem here and two problems there," he added. "What we don't see is 75 or 100 people in the environment all writing different programs and using it for this big cluster. We don't see it providing the real economies of scale that it should at the data-center level."

Bird acknowledged that a lot of vendors are building cloud-based solutions for security, analytics, and other features on top of Hadoop, a platform that Bird claims breaks one of the "cardinal rules" of software engineering.

"You don't try to build security onto an application," he said. "You need to think about it from the ground up."

Another of Hadoop's shortcomings is that it provides very basic levels of security, he said.

"A lot of enterprise security controls that you would expect, and are familiar with from basic Windows Server deployments, are simply not available on Hadoop clusters," Bird said.

Organizations' lack of attention to Hadoop security is another cause for concern.

"From our experience, most people don't realize they have to secure Hadoop, which is probably a little scarier," said Bird.

Presentations at the Black Hat and Defcon conferences in the past two years have exposed Hadoop's security shortcomings, but more education is needed, he said.

"What I'm concerned about with regards to Hadoop is that a lot of companies are (starting) very serious data centralization efforts where they're bringing a lot of data into one place, and yet it's the most insecure place in their entire environment," Bird said.

Despite the framework's security inadequacies, there has yet to be a highly publicized security breach that's specifically Hadoop-related. The platform's growing popularity could change that. Bird said: "I would expect, quite frankly, that as adoption climbs, that will be the case."

Even if today's enterprise breaches can't be attributed directly to Hadoop, the platform "isn't providing the level of security that one would want to deploy," said Red Lambda executive vice president William Ronca.

Red Lambda officials do expect to see improvements in Hadoop's security, but not until developers make a concerted effort to do so. "I think the real concern is how long that will take, and how much data gets pushed into Hadoop clusters over the next two or three years while people figure out how to secure Hadoop," Bird said.

Big data places heavy demands on storage infrastructure. In the new, all-digital Big Storage issue of InformationWeek Government, find out how federal agencies must adapt their architectures and policies to optimize it all. Also, we explain why tape storage continues to survive and thrive.