Big Data // Big Data Analytics
Commentary
7/2/2014
08:44 AM
Connect Directly
RSS
E-Mail
0%
100%

Hadoop Jobs: How To Outsmart Unicorn Hunters

While some employers insist on searching for mythical perfect candidates, mere mortals can win Hadoop jobs. Search wisely and demonstrate the right blend of skills.

In a Shel Silverstein kids' song the unicorns ignored the warnings of rain and missed the boat as Noah's Ark sailed away.

Today, despite a couple of years of warnings from experts, employers are wasting time searching for their own unicorns: all-knowing data scientists who can run big data initiatives.

These unicorns might lack the traditional forehead appendage but they are expected to bring to the table great technical chops, including not only Hadoop development and architectural expertise, but analytical skills, problem solving ability, business/industry experience, leadership talent, and great communication skills. (It probably doesn't hurt if they also can cook and sing.)

[Data scientist jobs are changing. Read Data Scientist Role: How Will It Evolve?]

"This happens whenever there is a new tech paradigm, employers always want that unicorn type of person. At that time people are still exploring and they eventually realize they need more than one person to fill those roles," said Nixon Patel, vice president with staffing company Collabera, in a recent phone interview. "However, after a couple of years they might be able to create that unicorn themselves, with one person learning much from all of the team members."

In the Hadoop sector, IT pros might not need deep experience in a half dozen highly diverse skill areas but the job postings floating around on the web make it clear that from staff-level developer to director-level data scientist, employers want Hadoop pros with a range of experiences. Not all of those skills will be readily thought of as big data related, but job candidates might be smart to include a decent mix at the top of their resumes.

Take some of the postings on the job site Dice.com, for example. A listing for a systems administrator, a classic IT title, carries the additional heading of "Hadoop administrator." It calls for five years of Hadoop management work, strong Linux and systems management experience, knowledge of multiple big data oriented tools, and storage experience.

Another listing for a "Hadoop application developer technician" -- carrying a salary in the $120,000 range -- calls for analytical and problem solving skills, two years of experience with Hadoop and other big data tools, work with Linux and other open source software systems, ETL experience, and sound knowledge of databases.

A listing for a "Hadoop senior architect" calls for at least five years of customer-facing experience as a consultant, two years of experience deploying large-scale Hadoop environments, data transformation experience with Apache PIG, and knowledge of MapReduce, all of which fall into the Hadoop realm. Yet, the job also requires more traditional IT experience with Linux or Unix, multiple enterprise security solutions, understanding of networking, and a "solid" background in database administration and design.

Often, job seekers have to look beyond job titles to find entry to the roles they want. Searches for "Hadoop" on Dice.com and Robert Half Technology's website revealed that about one third of the results returned didn't mention Hadoop in the job title. Hadoop fell under titles such as Backend Software Engineer, Vice President of Data Science, Linux Engineer, BI Consultant, Data Warehousing ETL Engineer, and Application Architect.

In a recent interview, Dice president Shravan Goli advised job seekers in the Hadoop space to highlight their skills beyond just Hadoop, such as experience with Java and NoSQL. "If you are looking to get a higher paying job in the tech field and have any of these big data skills, you can train yourself to be a Hadoop developer and become more valuable," he said. He noted that because of the scarcity of Hadoop qualified candidates, companies are becoming more creative and flexible in who they hire.

The Hadoop concept itself calls for systems management and clustering skills, an understanding of data management, and problem-solving capabilities, along with other traditional IT skills. When a job candidate layers on the Hadoop-specific skillset, it might not add up to a unicorn, but at least a horse of a different color.

Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift. Get the 6 Tools To Protect Big Data report today (registration required).

Jim Connolly is a versatile and experienced technology journalist who has reported on IT trends for more than two decades. He has written about enterprise computing, the PC revolution, client/server, the evolution of the Internet, networking, IT management, and the ongoing ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Page 1 / 2   >   >>
Jamescon
50%
50%
Jamescon,
User Rank: Moderator
7/14/2014 | 8:11:26 AM
Re: Hadoop experience
@Joe. While I agree that overreliance on software has changed HR's performance for the worse, I would stop short of saying the departments have outlived their usefulness. I'd suggest that they and senior management re-evaluate HR's role in an individual company, define what needs to be done in a centralized role and the best way to achieve those goals.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Ninja
7/13/2014 | 8:21:16 AM
Re: Hadoop experience
Indeed, James.  Why are we still paying HR staffers???  It seems that the profession has outlived its usefulness ever since they stopped adding value and started relying on dumb "smart" software.

(And as for the compliance stuff -- all things that could easily (and much more effectively, in my experience) be taken care of by in-house counsel.)
Jamescon
50%
50%
Jamescon,
User Rank: Moderator
7/10/2014 | 7:29:13 AM
Re: Hadoop experience
@Joe Stanganelli. That sounds about right in terms of how people get/don't get hired.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Ninja
7/10/2014 | 7:11:33 AM
Re: Hadoop experience
@James: In other cases, HR outright pressures hiring managers to hammer down a "years" requirement -- even where the hiring manager thinks it's not particularly important.

Result: Of the hundreds of resumes received, 99% are filtered out (thanks to HR's keyword-narrowing software so they don't have to, you know, do their jobs and actually look at a resume), then a few are read, an even fewer are sent to the hiring manager -- but the hiring manager at that point has already decided to hire a clever networker who knows someone who knows him.
Jamescon
50%
50%
Jamescon,
User Rank: Moderator
7/7/2014 | 12:25:20 PM
Re: Hadoop experience
@Danielcawrey. Right, HR loves to assign "years" of experience to a job post. In a case like big data or Hadoop where the years requirement exceeds the time that the tech/buzzword has been in existance, you end up with one of two things happening. Applicants have to apply new buzzwords to old, less glamorous experiences (that seems fair). Or, they outright lie (not the desired result). 
Jamescon
50%
50%
Jamescon,
User Rank: Moderator
7/7/2014 | 12:20:13 PM
Re: Hadoop experience
That would meet the HR department's need for a skills/experience checklist. They probably would accept that without question.
Laurianne
50%
50%
Laurianne,
User Rank: Author
7/7/2014 | 11:31:59 AM
Re: Hadoop experience
@Joe, well-played, that tweet is a perfect fit here. I think both recruiters and job hunters realize how crazy our "wants" in the IT job descriptions have become.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Ninja
7/6/2014 | 8:06:40 AM
Re: Hadoop experience
@Laurianne: Reminds me of this Tweet that came out during Apple WWDC when Apple's new programming language, Swift, was announced: https://twitter.com/iamdevloper/status/473537201125736448
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Ninja
7/6/2014 | 8:03:59 AM
About to break its neck???
I almost wish that picture hadn't been chosen with that headline... It really makes the girl in the painting look menacing now.
danielcawrey
50%
50%
danielcawrey,
User Rank: Ninja
7/5/2014 | 4:33:59 PM
Re: Hadoop experience
I was thinking the exact same thing about the amount of experience. It is interesting that HR would put that amount of experience in a job posting. Where do they expect someone to get that much experience. You would have had to be pretty early in the game, which is a tiny number of people. 

That being said, I think having any sort of big data experience over the next few years is going to be relevant enough for these types of job postings. 
Page 1 / 2   >   >>
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - August 20, 2014
CIOs need people who know the ins and outs of cloud software stacks and security, and, most of all, can break through cultural resistance.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.