Hadoop Jobs: Key Skills For Advancement - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Big Data Analytics

Hadoop Jobs: Key Skills For Advancement

Many people want lucrative Hadoop jobs right now. Do you have the Hadoop expertise and experience necessary to keep your career moving forward?

Hadoop Jobs: 9 Ways To Get Hired
Hadoop Jobs: 9 Ways To Get Hired
(Click image for larger view and slideshow.)

In the midst of an era when adoption of big data analytics is expected to generate more than 4 million jobs worldwide between 2013 and 2016, an IT professional might think that dropping the name "Hadoop" is a ticket to a salary bump, a promotion, or a new job with a new company.

Of course, they would be wrong. Employers increasingly are looking for just the right Hadoop-related skillsets, according to Nixon Patel, VP in charge of the Emerging Technology Competency Unit for staffing firm Collabera.

The Hadoop field is going through a transition on multiple fronts, and IT pros need to know where the technology is heading and which skills will be in demand over the years to come, according to Patel. For example, enterprises are shifting from the original batch-oriented Hadoop implementations to more of a real-time model. That opens the door for use of Apache Spark, designed to run 10 to 100 times faster than MapReduce, which has been in use for several years. So, advantage to Hadoop pros with Spark experience. Even those who are experienced with MapReduce want to be familiar with the newest version, Yarn, said Patel.

Nixon Patel
Nixon Patel

Besides having experience with emerging Hadoop tools, the staff-level IT pro who is in the best position to advance has five to seven years of IT experience. They should display analytical abilities and, Patel added, a "zeal and passion for learning new things and to want to commit some time.

[What is it worth to move ahead in the big data field? Read $60,000 Online Degree: A Lesson In Digital Business.]

"Having the name [Hadoop] on their resume is not going to be getting them far. They need to have hands-on experience in terms of how the architecture is going to work," said Patel, who heads a Collabera "train and deploy" program that trains IT pros for several months in emerging technologies.

Another approach for mid-career IT pros who want to advance with their current employers is to identify which Hadoop distribution their company wants to use. Whether they opt for Hortonworks, Cloudera, MapR, or another distribution, IT pros can position themselves as Hadoop pros.

However, the real bonus in doing so, according to data scientist Dan Gutierrez of Amulet Analytics, is that the price is just right for training on one distribution, free.

IT professionals and their employers are accustomed to paying anywhere from $150 to more than $700 for training and certification in areas such as Windows administration and networking. "Hadoop companies know that's a model they can't follow. So they are giving away a lot of resources. They do have for-fee training, but they also have a lot of resources for free," said Gutierrez in a phone interview.

He noted that companies providing Hadoop distributions need a core of specialists at customer sites who understand the architectures and languages of their particular software. In addition to providing free training, it's not unusual for the software companies to punctuate training and even appearances at local tech meet-ups by telling IT pros that they themselves are hiring, said Gutierrez, who teaches Hadoop courses through Coursera. So, the free training exposes the software companies to newly qualified job candidates.

He said experienced IT professionals can guide their employers into the world of big data by taking training courses and then setting up pilot, proof-of-concept applications on a public cloud infrastructure. With that skunkworks-style approach, there is no in-house infrastructure investment.

This isn't an overnight exercise, however, as Gutierrez said that IT pros who want to get into Hadoop and big data analytics should allow a year for training and building out a pilot project for their current company.

In addition to having real-world Hadoop experience, Collabera's Patel suggests that an IT pro who wants to advance their career should have experience with some of the core tools that go into Hadoop development and administration. He cited Python, Java, and R as examples, along with an interest in machine learning. One advantage that an experienced IT pro does have when moving into Hadoop, he said, is that the basic Hadoop commands have their roots in Unix and Linux, which many IT pros already know. That experience with legacy systems can help to position them for growth with today's emerging technology.

InformationWeek's June Must Reads is a compendium of our best recent coverage of big data. Find out one CIO's take on what's driving big data, key points on platform considerations, why a recent White House report on the topic has earned praise and skepticism, and much more.

Jim Connolly is a versatile and experienced technology journalist who has reported on IT trends for more than two decades. As editorial director of InformationWeek and Network Computing, he oversees the day-to-day planning and editing on the site. Most recently he was editor ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Jamescon
50%
50%
Jamescon,
User Rank: Author
7/2/2014 | 9:51:06 AM
Re: Future will reside on adapters and plug-ins
@Mahender. My suspicion is that while there always will be some dedicated analytical apps, another growth area (maybe the biggest) for analytics will be as an element in everyday business applications. I think it has something in common with AI in that regard (which also includes the machine learning that is an element in big data). Yes, there are specific AI apps, but AI also appears in most of the software that we use today. We don't think of it that way, but isn't spell/grammar check in Word essentially an AI app in that it learns from our behavior (as individuals and in aggregate)? Search falls into the same category.

Suppose that we applied analytics in the background as we engaged in email, social media, and document exchanges with someone in our workplace, and the results told us more about them and our history with them while we were interacting with them. Is that the future for Outlook, Gmail or other commercial packages?

 
Jamescon
50%
50%
Jamescon,
User Rank: Author
7/1/2014 | 12:23:54 PM
Re: Leading edge technology
Good catch!
Jamescon
50%
50%
Jamescon,
User Rank: Author
7/1/2014 | 11:13:53 AM
Re: Future will reside on adapters and plug-ins
Mahender. You're right about Hadoop and related technologies being crucial as we look toward the future. I'm curious: Do you see a day when we not only stop referring to "big data" and simply think in terms of analytics (regardless of data's size) and where analytics capabilities aren't delivered as independent applications but are simply sitting under the covers of all/most general business apps?

 
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

Slideshows
11 Things IT Professionals Wish They Knew Earlier in Their Careers
Lisa Morgan, Freelance Writer,  4/6/2021
News
Time to Shift Your Job Search Out of Neutral
Jessica Davis, Senior Editor, Enterprise Apps,  3/31/2021
Commentary
Does Identity Hinder Hybrid-Cloud and Multi-Cloud Adoption?
Joao-Pierre S. Ruth, Senior Writer,  4/1/2021
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Successful Strategies for Digital Transformation
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Slideshows
Flash Poll