Big Data // Big Data Analytics
11:24 AM
Connect Directly
Repost This

Data Scientist Shortage: Split Role In Half

Splitting the role of data scientist into two jobs -- data manager and data analyst -- is one way to solve the expected shortage of information gurus, says one technology strategist.

13 Big Data Vendors To Watch In 2013
13 Big Data Vendors To Watch In 2013
(click image for larger view and for slideshow)
"Big Data Creates Big Jobs" trumpeted a recent news headline from research firm Gartner, which predicts that 4.4 million IT jobs will be created globally by 2015 to support big data operations, including 1.9 million IT jobs in the United States.

Sounds great, right? Not necessarily.

"There is not enough talent in the industry. Our public and private education systems are failing us. Therefore, only one-third of the IT jobs will be filled. Data experts will be a scarce, valuable commodity," said Peter Sondergaard, Gartner senior vice president and global head of research, in a statement.

One possible solution to this staffing shortage is to split big data duties in two: data management specialist and data scientist. In a phone interview with InformationWeek, Datalink technology strategist and consulting principal Steve Bulmer said this division of labor would help IT organizations prepare for the coming "tidal wave" of data. "Advanced analytic capabilities are going to be in high demand and hard to find," said Bulmer. "And that's one reason I think that big data -- and the roles that are required to manage and analyze big data -- are going to be split."

[ For another perspective on handling the dearth of big data experts, see Big Data And Analytics Expertise: Beg, Borrow Or Steal? ]

Bulmer is former chief technical officer of Strategic Technologies, or StraTech, a small IT services firm bought in October 2012 by Datalink, which provides data center services to large to medium enterprises. In a recent blog post on big data, Bulmer predicted that the era of the data scientist's "secret domain" is rapidly coming to an end, as organizations try new approaches to manage ever-growing volumes of unstructured information from a variety of sources.

"This is my tsunami warning to IT organizations and providers: IT departments will be called upon more and more to handle the data management phase in order to offload this effort from that scarce data scientist resource; this shift will be the result of concentrating the scientists' focus on the analytics, visualization and actual business absorption of the analytical results," Bulmer wrote.

In fact, the tsunami is just about to hit. In 2016, two-thirds of the mobile workforce will own a smartphone, and global consumers will buy more than 1.6 billion smart mobile devices, Gartner predicts. Those devices, combined with a dramatic increase in machine-to-machine transactions, will mean greater responsibilities for IT staff. "Next year we'll see an awareness from IT professionals that they're going to be called upon to be the information stewards in the big data process," Bulmer told InformationWeek.

The staffing shortages are increasingly easy to spot. When Bulmer attended the Hadoop World trade show in October, for instance, he noticed that the bulletin boards there were packed with postings from startups and other companies seeking data scientists.

Although more business schools are offering analytics courses to provide university graduates with a basic set of data science skills, a looming talent shortage is inevitable.

"The data scientists are going to be lacking for a while, that's for sure," said Bulmer. He cited a recent New York Times article that explains how online retailers are using sophisticated algorithms to change prices hourly to stay ahead of the competition -- projects that demand big-data gurus. "I think more and more data is going to be sought after for competitive advantage," Bulmer said. "And I think it's happening now."

Predictive analysis is getting faster, more accurate and more accessible. Combined with big data, it's driving a new age of experiments. Also in the new, all-digital Advanced Analytics issue of InformationWeek: Are project management offices a waste of money? (Free registration required.)

Comment  | 
Print  | 
More Insights
InformationWeek Elite 100
InformationWeek Elite 100
Our data shows these innovators using digital technology in two key areas: providing better products and cutting costs. Almost half of them expect to introduce a new IT-led product this year, and 46% are using technology to make business processes more efficient.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.