Big Data // Big Data Analytics
Commentary
7/2/2014
08:44 AM
Connect Directly
RSS
E-Mail
0%
100%

Hadoop Jobs: How To Outsmart Unicorn Hunters

While some employers insist on searching for mythical perfect candidates, mere mortals can win Hadoop jobs. Search wisely and demonstrate the right blend of skills.

In a Shel Silverstein kids' song the unicorns ignored the warnings of rain and missed the boat as Noah's Ark sailed away.

Today, despite a couple of years of warnings from experts, employers are wasting time searching for their own unicorns: all-knowing data scientists who can run big data initiatives.

These unicorns might lack the traditional forehead appendage but they are expected to bring to the table great technical chops, including not only Hadoop development and architectural expertise, but analytical skills, problem solving ability, business/industry experience, leadership talent, and great communication skills. (It probably doesn't hurt if they also can cook and sing.)

[Data scientist jobs are changing. Read Data Scientist Role: How Will It Evolve?]

"This happens whenever there is a new tech paradigm, employers always want that unicorn type of person. At that time people are still exploring and they eventually realize they need more than one person to fill those roles," said Nixon Patel, vice president with staffing company Collabera, in a recent phone interview. "However, after a couple of years they might be able to create that unicorn themselves, with one person learning much from all of the team members."

In the Hadoop sector, IT pros might not need deep experience in a half dozen highly diverse skill areas but the job postings floating around on the web make it clear that from staff-level developer to director-level data scientist, employers want Hadoop pros with a range of experiences. Not all of those skills will be readily thought of as big data related, but job candidates might be smart to include a decent mix at the top of their resumes.

Take some of the postings on the job site Dice.com, for example. A listing for a systems administrator, a classic IT title, carries the additional heading of "Hadoop administrator." It calls for five years of Hadoop management work, strong Linux and systems management experience, knowledge of multiple big data oriented tools, and storage experience.

Another listing for a "Hadoop application developer technician" -- carrying a salary in the $120,000 range -- calls for analytical and problem solving skills, two years of experience with Hadoop and other big data tools, work with Linux and other open source software systems, ETL experience, and sound knowledge of databases.

A listing for a "Hadoop senior architect" calls for at least five years of customer-facing experience as a consultant, two years of experience deploying large-scale Hadoop environments, data transformation experience with Apache PIG, and knowledge of MapReduce, all of which fall into the Hadoop realm. Yet, the job also requires more traditional IT experience with Linux or Unix, multiple enterprise security solutions, understanding of networking, and a "solid" background in database administration and design.

Often, job seekers have to look beyond job titles to find entry to the roles they want. Searches for "Hadoop" on Dice.com and Robert Half Technology's website revealed that about one third of the results returned didn't mention Hadoop in the job title. Hadoop fell under titles such as Backend Software Engineer, Vice President of Data Science, Linux Engineer, BI Consultant, Data Warehousing ETL Engineer, and Application Architect.

In a recent interview, Dice president Shravan Goli advised job seekers in the Hadoop space to highlight their skills beyond just Hadoop, such as experience with Java and NoSQL. "If you are looking to get a higher paying job in the tech field and have any of these big data skills, you can train yourself to be a Hadoop developer and become more valuable," he said. He noted that because of the scarcity of Hadoop qualified candidates, companies are becoming more creative and flexible in who they hire.

The Hadoop concept itself calls for systems management and clustering skills, an understanding of data management, and problem-solving capabilities, along with other traditional IT skills. When a job candidate layers on the Hadoop-specific skillset, it might not add up to a unicorn, but at least a horse of a different color.

Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift. Get the 6 Tools To Protect Big Data report today (registration required).

Jim Connolly is a versatile and experienced technology journalist who has reported on IT trends for more than two decades. He has written about enterprise computing, the PC revolution, client/server, the evolution of the Internet, networking, IT management, and the ongoing ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
<<   <   Page 2 / 2
Laurianne
50%
50%
Laurianne,
User Rank: Author
7/3/2014 | 12:35:47 PM
Re: Balance and triage
Good point, "soft" skills like communication skills are important to demonstrate. You can analyze the data but can you explain it?
Laurianne
50%
50%
Laurianne,
User Rank: Author
7/2/2014 | 10:45:47 AM
Hadoop experience
Job postings seeking five years of Hadoop management experience seem a little unrealistic. Many people who are passionate about big data have only been doing it for two or three years. What I heard from big data leaders at a recent conference was don't let the job descriptions scare you: Work your way into the interview and demonstrate related IT skills and curiosity.
Li Tan
50%
50%
Li Tan,
User Rank: Ninja
7/2/2014 | 9:14:59 AM
Balance and triage
I do agree with the points here. To hunt for hadoop  job, you do not need to be a top talent in this planet. You don't need to be a mathematician except you are doing pure research work in the tough area such as algorithm. Instead you should possess the necessary skillset and have balance among different skills - both technical and soft skill. Most of the employers will have pragmatic choice and balance pros/cons of the candidate. The same holds true not only for Hadoop jobs but for other positions as wel.
<<   <   Page 2 / 2
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest September 18, 2014
Enterprise social network success starts and ends with integration. Here's how to finally make collaboration click.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.