Splitting the role of data scientist into two jobs -- data manager and data analyst -- is one way to solve the expected shortage of information gurus, says one technology strategist.
13 Big Data Vendors To Watch In 2013
(click image for larger view and for slideshow)
"Big Data Creates Big Jobs" trumpeted a recent news headline from research firm Gartner, which predicts that 4.4 million IT jobs will be created globally by 2015 to support big data operations, including 1.9 million IT jobs in the United States.
Sounds great, right? Not necessarily.
"There is not enough talent in the industry. Our public and private education systems are failing us. Therefore, only one-third of the IT jobs will be filled. Data experts will be a scarce, valuable commodity," said Peter Sondergaard, Gartner senior vice president and global head of research, in a statement.
One possible solution to this staffing shortage is to split big data duties in two: data management specialist and data scientist. In a phone interview with InformationWeek, Datalink technology strategist and consulting principal Steve Bulmer said this division of labor would help IT organizations prepare for the coming "tidal wave" of data. "Advanced analytic capabilities are going to be in high demand and hard to find," said Bulmer. "And that's one reason I think that big data -- and the roles that are required to manage and analyze big data -- are going to be split."
Bulmer is former chief technical officer of Strategic Technologies, or StraTech, a small IT services firm bought in October 2012 by Datalink, which provides data center services to large to medium enterprises. In a recent blog post on big data, Bulmer predicted that the era of the data scientist's "secret domain" is rapidly coming to an end, as organizations try new approaches to manage ever-growing volumes of unstructured information from a variety of sources.
"This is my tsunami warning to IT organizations and providers: IT departments will be called upon more and more to handle the data management phase in order to offload this effort from that scarce data scientist resource; this shift will be the result of concentrating the scientists' focus on the analytics, visualization and actual business absorption of the analytical results," Bulmer wrote.
In fact, the tsunami is just about to hit. In 2016, two-thirds of the mobile workforce will own a smartphone, and global consumers will buy more than 1.6 billion smart mobile devices, Gartner predicts. Those devices, combined with a dramatic increase in machine-to-machine transactions, will mean greater responsibilities for IT staff. "Next year we'll see an awareness from IT professionals that they're going to be called upon to be the information stewards in the big data process," Bulmer told InformationWeek.
The staffing shortages are increasingly easy to spot. When Bulmer attended the Hadoop World trade show in October, for instance, he noticed that the bulletin boards there were packed with postings from startups and other companies seeking data scientists.
Although more business schools are offering analytics courses to provide university graduates with a basic set of data science skills, a looming talent shortage is inevitable.
"The data scientists are going to be lacking for a while, that's for sure," said Bulmer. He cited a recent New York Times article that explains how online retailers are using sophisticated algorithms to change prices hourly to stay ahead of the competition -- projects that demand big-data gurus. "I think more and more data is going to be sought after for competitive advantage," Bulmer said. "And I think it's happening now."
Predictive analysis is getting faster, more accurate and more accessible. Combined with big data, it's driving a new age of experiments. Also in the new, all-digital Advanced Analytics issue of InformationWeek: Are project management offices a waste of money? (Free registration required.)
How Enterprises Are Attacking the IT Security EnterpriseTo learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
2017 State of IT ReportIn today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.