Hadoop Jobs: How To Outsmart Unicorn Hunters - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Big Data Analytics
08:44 AM

Hadoop Jobs: How To Outsmart Unicorn Hunters

While some employers insist on searching for mythical perfect candidates, mere mortals can win Hadoop jobs. Search wisely and demonstrate the right blend of skills.

In a Shel Silverstein kids' song the unicorns ignored the warnings of rain and missed the boat as Noah's Ark sailed away.

Today, despite a couple of years of warnings from experts, employers are wasting time searching for their own unicorns: all-knowing data scientists who can run big data initiatives.

These unicorns might lack the traditional forehead appendage but they are expected to bring to the table great technical chops, including not only Hadoop development and architectural expertise, but analytical skills, problem solving ability, business/industry experience, leadership talent, and great communication skills. (It probably doesn't hurt if they also can cook and sing.)

[Data scientist jobs are changing. Read Data Scientist Role: How Will It Evolve?]

"This happens whenever there is a new tech paradigm, employers always want that unicorn type of person. At that time people are still exploring and they eventually realize they need more than one person to fill those roles," said Nixon Patel, vice president with staffing company Collabera, in a recent phone interview. "However, after a couple of years they might be able to create that unicorn themselves, with one person learning much from all of the team members."

In the Hadoop sector, IT pros might not need deep experience in a half dozen highly diverse skill areas but the job postings floating around on the web make it clear that from staff-level developer to director-level data scientist, employers want Hadoop pros with a range of experiences. Not all of those skills will be readily thought of as big data related, but job candidates might be smart to include a decent mix at the top of their resumes.

Take some of the postings on the job site Dice.com, for example. A listing for a systems administrator, a classic IT title, carries the additional heading of "Hadoop administrator." It calls for five years of Hadoop management work, strong Linux and systems management experience, knowledge of multiple big data oriented tools, and storage experience.

Another listing for a "Hadoop application developer technician" -- carrying a salary in the $120,000 range -- calls for analytical and problem solving skills, two years of experience with Hadoop and other big data tools, work with Linux and other open source software systems, ETL experience, and sound knowledge of databases.

A listing for a "Hadoop senior architect" calls for at least five years of customer-facing experience as a consultant, two years of experience deploying large-scale Hadoop environments, data transformation experience with Apache PIG, and knowledge of MapReduce, all of which fall into the Hadoop realm. Yet, the job also requires more traditional IT experience with Linux or Unix, multiple enterprise security solutions, understanding of networking, and a "solid" background in database administration and design.

Often, job seekers have to look beyond job titles to find entry to the roles they want. Searches for "Hadoop" on Dice.com and Robert Half Technology's website revealed that about one third of the results returned didn't mention Hadoop in the job title. Hadoop fell under titles such as Backend Software Engineer, Vice President of Data Science, Linux Engineer, BI Consultant, Data Warehousing ETL Engineer, and Application Architect.

In a recent interview, Dice president Shravan Goli advised job seekers in the Hadoop space to highlight their skills beyond just Hadoop, such as experience with Java and NoSQL. "If you are looking to get a higher paying job in the tech field and have any of these big data skills, you can train yourself to be a Hadoop developer and become more valuable," he said. He noted that because of the scarcity of Hadoop qualified candidates, companies are becoming more creative and flexible in who they hire.

The Hadoop concept itself calls for systems management and clustering skills, an understanding of data management, and problem-solving capabilities, along with other traditional IT skills. When a job candidate layers on the Hadoop-specific skillset, it might not add up to a unicorn, but at least a horse of a different color.

Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift. Get the 6 Tools To Protect Big Data report today (registration required).

Jim Connolly is a versatile and experienced technology journalist who has reported on IT trends for more than two decades. As editorial director of InformationWeek and Network Computing, he oversees the day-to-day planning and editing on the site. Most recently he was editor ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
<<   <   Page 2 / 2
User Rank: Author
7/3/2014 | 12:35:47 PM
Re: Balance and triage
Good point, "soft" skills like communication skills are important to demonstrate. You can analyze the data but can you explain it?
User Rank: Author
7/2/2014 | 10:45:47 AM
Hadoop experience
Job postings seeking five years of Hadoop management experience seem a little unrealistic. Many people who are passionate about big data have only been doing it for two or three years. What I heard from big data leaders at a recent conference was don't let the job descriptions scare you: Work your way into the interview and demonstrate related IT skills and curiosity.
Li Tan
Li Tan,
User Rank: Ninja
7/2/2014 | 9:14:59 AM
Balance and triage
I do agree with the points here. To hunt for hadoop  job, you do not need to be a top talent in this planet. You don't need to be a mathematician except you are doing pure research work in the tough area such as algorithm. Instead you should possess the necessary skillset and have balance among different skills - both technical and soft skill. Most of the employers will have pragmatic choice and balance pros/cons of the candidate. The same holds true not only for Hadoop jobs but for other positions as wel.
<<   <   Page 2 / 2
The State of Chatbots: Pandemic Edition
Jessica Davis, Senior Editor, Enterprise Apps,  9/10/2020
Deloitte on Cloud, the Edge, and Enterprise Expectations
Joao-Pierre S. Ruth, Senior Writer,  9/14/2020
Data Science: How the Pandemic Has Affected 10 Popular Jobs
Cynthia Harvey, Freelance Journalist, InformationWeek,  9/9/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
IT Automation Transforms Network Management
In this special report we will examine the layers of automation and orchestration in IT operations, and how they can provide high availability and greater scale for modern applications and business demands.
Flash Poll