Big Data // Big Data Analytics
News
12/11/2012
11:24 AM
Connect Directly
Google+
RSS
E-Mail
50%
50%

Data Scientist Shortage: Split Role In Half

Splitting the role of data scientist into two jobs -- data manager and data analyst -- is one way to solve the expected shortage of information gurus, says one technology strategist.

13 Big Data Vendors To Watch In 2013
13 Big Data Vendors To Watch In 2013
(click image for larger view and for slideshow)
"Big Data Creates Big Jobs" trumpeted a recent news headline from research firm Gartner, which predicts that 4.4 million IT jobs will be created globally by 2015 to support big data operations, including 1.9 million IT jobs in the United States.

Sounds great, right? Not necessarily.

"There is not enough talent in the industry. Our public and private education systems are failing us. Therefore, only one-third of the IT jobs will be filled. Data experts will be a scarce, valuable commodity," said Peter Sondergaard, Gartner senior vice president and global head of research, in a statement.

One possible solution to this staffing shortage is to split big data duties in two: data management specialist and data scientist. In a phone interview with InformationWeek, Datalink technology strategist and consulting principal Steve Bulmer said this division of labor would help IT organizations prepare for the coming "tidal wave" of data. "Advanced analytic capabilities are going to be in high demand and hard to find," said Bulmer. "And that's one reason I think that big data -- and the roles that are required to manage and analyze big data -- are going to be split."

[ For another perspective on handling the dearth of big data experts, see Big Data And Analytics Expertise: Beg, Borrow Or Steal? ]

Bulmer is former chief technical officer of Strategic Technologies, or StraTech, a small IT services firm bought in October 2012 by Datalink, which provides data center services to large to medium enterprises. In a recent blog post on big data, Bulmer predicted that the era of the data scientist's "secret domain" is rapidly coming to an end, as organizations try new approaches to manage ever-growing volumes of unstructured information from a variety of sources.

"This is my tsunami warning to IT organizations and providers: IT departments will be called upon more and more to handle the data management phase in order to offload this effort from that scarce data scientist resource; this shift will be the result of concentrating the scientists' focus on the analytics, visualization and actual business absorption of the analytical results," Bulmer wrote.

In fact, the tsunami is just about to hit. In 2016, two-thirds of the mobile workforce will own a smartphone, and global consumers will buy more than 1.6 billion smart mobile devices, Gartner predicts. Those devices, combined with a dramatic increase in machine-to-machine transactions, will mean greater responsibilities for IT staff. "Next year we'll see an awareness from IT professionals that they're going to be called upon to be the information stewards in the big data process," Bulmer told InformationWeek.

The staffing shortages are increasingly easy to spot. When Bulmer attended the Hadoop World trade show in October, for instance, he noticed that the bulletin boards there were packed with postings from startups and other companies seeking data scientists.

Although more business schools are offering analytics courses to provide university graduates with a basic set of data science skills, a looming talent shortage is inevitable.

"The data scientists are going to be lacking for a while, that's for sure," said Bulmer. He cited a recent New York Times article that explains how online retailers are using sophisticated algorithms to change prices hourly to stay ahead of the competition -- projects that demand big-data gurus. "I think more and more data is going to be sought after for competitive advantage," Bulmer said. "And I think it's happening now."

Predictive analysis is getting faster, more accurate and more accessible. Combined with big data, it's driving a new age of experiments. Also in the new, all-digital Advanced Analytics issue of InformationWeek: Are project management offices a waste of money? (Free registration required.)

Comment  | 
Print  | 
More Insights
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Government Tech Digest Oct. 27, 2014
To meet obligations -- and avoid accusations of cover-up and incompetence -- federal agencies must get serious about digitizing records.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of October 26, 2014 and for the incredible Friday Afternoon Conversation that runs beside the program.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.