Big Data // Big Data Analytics
News
6/27/2014
09:06 AM
Connect Directly
RSS
E-Mail
50%
50%

Hadoop Jobs: Key Skills For Advancement

Many people want lucrative Hadoop jobs right now. Do you have the Hadoop expertise and experience necessary to keep your career moving forward?

Hadoop Jobs: 9 Ways To Get Hired
Hadoop Jobs: 9 Ways To Get Hired
(Click image for larger view and slideshow.)

In the midst of an era when adoption of big data analytics is expected to generate more than 4 million jobs worldwide between 2013 and 2016, an IT professional might think that dropping the name "Hadoop" is a ticket to a salary bump, a promotion, or a new job with a new company.

Of course, they would be wrong. Employers increasingly are looking for just the right Hadoop-related skillsets, according to Nixon Patel, VP in charge of the Emerging Technology Competency Unit for staffing firm Collabera.

The Hadoop field is going through a transition on multiple fronts, and IT pros need to know where the technology is heading and which skills will be in demand over the years to come, according to Patel. For example, enterprises are shifting from the original batch-oriented Hadoop implementations to more of a real-time model. That opens the door for use of Apache Spark, designed to run 10 to 100 times faster than MapReduce, which has been in use for several years. So, advantage to Hadoop pros with Spark experience. Even those who are experienced with MapReduce want to be familiar with the newest version, Yarn, said Patel.

Nixon Patel
Nixon Patel

Besides having experience with emerging Hadoop tools, the staff-level IT pro who is in the best position to advance has five to seven years of IT experience. They should display analytical abilities and, Patel added, a "zeal and passion for learning new things and to want to commit some time.

[What is it worth to move ahead in the big data field? Read $60,000 Online Degree: A Lesson In Digital Business.]

"Having the name [Hadoop] on their resume is not going to be getting them far. They need to have hands-on experience in terms of how the architecture is going to work," said Patel, who heads a Collabera "train and deploy" program that trains IT pros for several months in emerging technologies.

Another approach for mid-career IT pros who want to advance with their current employers is to identify which Hadoop distribution their company wants to use. Whether they opt for Hortonworks, Cloudera, MapR, or another distribution, IT pros can position themselves as Hadoop pros.

However, the real bonus in doing so, according to data scientist Dan Gutierrez of Amulet Analytics, is that the price is just right for training on one distribution, free.

IT professionals and their employers are accustomed to paying anywhere from $150 to more than $700 for training and certification in areas such as Windows administration and networking. "Hadoop companies know that's a model they can't follow. So they are giving away a lot of resources. They do have for-fee training, but they also have a lot of resources for free," said Gutierrez in a phone interview.

He noted that companies providing Hadoop distributions need a core of specialists at customer sites who understand the architectures and languages of their particular software. In addition to providing free training, it's not unusual for the software companies to punctuate training and even appearances at local tech meet-ups by telling IT pros that they themselves are hiring, said Gutierrez, who teaches Hadoop courses through Coursera. So, the free training exposes the software companies to newly qualified job candidates.

He said experienced IT professionals can guide their employers into the world of big data by taking training courses and then setting up pilot, proof-of-concept applications on a public cloud infrastructure. With that skunkworks-style approach, there is no in-house infrastructure investment.

This isn't an overnight exercise, however, as Gutierrez said that IT pros who want to get into Hadoop and big data analytics should allow a year for training and building out a pilot project for their current company.

In addition to having real-world Hadoop experience, Collabera's Patel suggests that an IT pro who wants to advance their career should have experience with some of the core tools that go into Hadoop development and administration. He cited Python, Java, and R as examples, along with an interest in machine learning. One advantage that an experienced IT pro does have when moving into Hadoop, he said, is that the basic Hadoop commands have their roots in Unix and Linux, which many IT pros already know. That experience with legacy systems can help to position them for growth with today's emerging technology.

InformationWeek's June Must Reads is a compendium of our best recent coverage of big data. Find out one CIO's take on what's driving big data, key points on platform considerations, why a recent White House report on the topic has earned praise and skepticism, and much more.

Jim Connolly is a versatile and experienced technology journalist who has reported on IT trends for more than two decades. He has written about enterprise computing, the PC revolution, client/server, the evolution of the Internet, networking, IT management, and the ongoing ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Jamescon
50%
50%
Jamescon,
User Rank: Moderator
7/2/2014 | 9:51:06 AM
Re: Future will reside on adapters and plug-ins
@Mahender. My suspicion is that while there always will be some dedicated analytical apps, another growth area (maybe the biggest) for analytics will be as an element in everyday business applications. I think it has something in common with AI in that regard (which also includes the machine learning that is an element in big data). Yes, there are specific AI apps, but AI also appears in most of the software that we use today. We don't think of it that way, but isn't spell/grammar check in Word essentially an AI app in that it learns from our behavior (as individuals and in aggregate)? Search falls into the same category.

Suppose that we applied analytics in the background as we engaged in email, social media, and document exchanges with someone in our workplace, and the results told us more about them and our history with them while we were interacting with them. Is that the future for Outlook, Gmail or other commercial packages?

 
Jamescon
50%
50%
Jamescon,
User Rank: Moderator
7/1/2014 | 12:23:54 PM
Re: Leading edge technology
Good catch!
Mahender
50%
50%
Mahender,
User Rank: Apprentice
7/1/2014 | 12:21:31 PM
Re: Future will reside on adapters and plug-ins
Jamescon, Yes I believe it will happen. Your thought on analytics as feature in future is highly convincing. My understanding - "Big Data" is a big selling point as of now for all service providers. Outcome of big data is analytics for a different data set. Few highly successful analytics companies are selling big data as analytics unlike service providers. As of now most of domain specializing service companies (or Cloud) also have analytics in place.Product companies might start initiative to provide analytics as integral part of their product. Though, it might take some time. Oracle has something around it currently which is being provided as a separate and known as ready to use data warehouse.mahender
burn0050bb
50%
50%
burn0050bb,
User Rank: Apprentice
7/1/2014 | 11:39:11 AM
Leading edge technology
Does anyone find it ironic that a guy talking about leading edge software has a CRT monitor on his desk?

 
Jamescon
50%
50%
Jamescon,
User Rank: Moderator
7/1/2014 | 11:13:53 AM
Re: Future will reside on adapters and plug-ins
Mahender. You're right about Hadoop and related technologies being crucial as we look toward the future. I'm curious: Do you see a day when we not only stop referring to "big data" and simply think in terms of analytics (regardless of data's size) and where analytics capabilities aren't delivered as independent applications but are simply sitting under the covers of all/most general business apps?

 
Mahender
50%
50%
Mahender,
User Rank: Apprentice
7/1/2014 | 2:15:32 AM
Future will reside on adapters and plug-ins
Future will reside on Hadoop and the adapters and plug-ins to support development around it. MY pov -Evolved ETL systems integrated with hadoop will change the big data landscape. mahender.
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - August 27, 2014
Who wins in cloud price wars? Short answer: not IT. Enterprises don't want bare-bones IaaS. Providers must focus on support, not undercutting rivals.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.