Big Data // Big Data Analytics
09:15 AM
Connect Directly
Repost This

Data Scientist: Hot Big Data Job

These specialists, who can supervise the integration of many types of data, find themselves in great demand compared to traditional data analysts.

Big Data Talent War: 10 Analytics Job Trends
Big Data Talent War: 10 Analytics Job Trends
(click image for larger view and for slideshow)
IT managers who have weathered a paradigm shift or two could be forgiven if they haven't taken to heart the many warnings of a critical, impending skills shortage in the market for big-data specialists.

Every major new technology that comes along--which happens about ever two or three years--causes a skills shortage in the IT hiring market, , according to Alice Hill, managing director of IT job-ad site

"There are a lot of perceived gaps in available skills in the new technology just because the technology is new and there hasn't been time for people with related skills to become trained in the new one," Hill said. "If a gap continues for a long time it could put upward pressure on salaries, but what usually happens is that companies end up training existing staff and only hire a few people with specialized skills."

The current shortage--or at least the perception of a shortage--becomes more intense when the overall market for IT people heats up, though both hiring and demand for IT skills has been relatively flat during the down economy of the past few years according to U.S. Bureau of Labor Statistics hiring figures.

[ Avoid common mistakes when launching a big data project. Read 10 Big Data Migration Mistakes. ]

The unemployment rate for technology workers is about half that of the general population --4.4% for tech workers during Q1 2012 compared to 8.3% for everyone else.

Within even tepid markets there are geographic or specialty areas in which the job situation is dramatically different than the average for the rest of the IT market, Hill said.

The hot, disruptive technologies of the moment--among which big data is the current leader--is the usual victim of such short-term shortages. And it appears that all big data skills are in short supply.

Mckinsey & Co., which published the seminal analysis of the growing big data market in 2011, predicted the U.S. would face a shortage of between 140,000 and 190,000 workers with the skills to manage and analyze big data.

On a national scale, those aren't large numbers. Compared to the existing supply of new graduates or existing specialists with the right skills, however, it represents a shortage of 50% to 60% even as far into the future as 2018.

Unlike most other disruptive technologies--cloud computing, virtualization, mobile computing, for example--big-data skills require specific talents with data, numbers, and business processes, and long training or retraining periods, McKinsey concluded.

The deeper the needed skills the more intense the talent shortage will be and the longer it will take to fill, the report concluded.

Though the McKinsey report was published in January of 2011, nothing has changed to lessen the intensity of the skills shortage in the time since, according to an April story in The Wall Street Journal quoting both big-data vendors and users.

Particularly sharp is the shortage not of big-data analysts, but data scientists--an imprecise term describing specialists who can not only supervise the integration of many types of data into a single data set and then find within it nuggets of information that could produce dramatic benefits.

Forty-five percent of business intelligence projects fail due to a lack of data expertise on staff, according to an April survey from GigaOm.

Because of the greater degree of difficulty gathering and integrating data, sifting it for answers, and translating the result into knowledge business units can use, it is even more important than in most data-centric projects to have real experts on staff for big-data projects than more run-of-the-mill efforts, according to Mike Boyarski, director of product marketing for business-intelligence/big-data software vendor Jaspersoft.

1 of 2
Comment  | 
Print  | 
More Insights
InformationWeek Elite 100
InformationWeek Elite 100
Our data shows these innovators using digital technology in two key areas: providing better products and cutting costs. Almost half of them expect to introduce a new IT-led product this year, and 46% are using technology to make business processes more efficient.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.