Should You Outsource Your Data Scientist?

13 Big Data Vendors To Watch In 2013

(click image for larger view and for slideshow)

The well-documented shortage of workers with advanced, data-science skills has sent nearly every organization on a mad dash to acquire the capability or be left behind.

Some have responded by trying to hire data scientists from the small, existing, expensive pool of talent. Others have sent current employees back to school, taking advantage of a blossoming number of university-level courses in data management and advanced analytics geared toward IT professionals already working in industry.

Unfortunately, neither approach is likely to fill the need. As Gartner recently estimated, only about one-third of global demand for big data-related jobs will be met.

But there is a third path: Outsourcing.

[ Read Outsourcing's New Reality: Choice Beats Cost. ]

"Look, [data scientists] are an extremely scarce resource," said Narendra Mulani, global managing director for Accenture Analytics in a phone interview. "Ideally, you should have it in-house, but is that feasible?"

On the other hand, Mulani and several other experts interviewed about the pros and cons of outsourcing the data-science function had strikingly similar advice. Although all said some aspects can be outsourced, all cautioned it was a mistake to think of these services as a commodity, given the degree of industry- and company-specific knowledge needed to derive actionable insights from complicated data models.

Moreover, every expert stressed the need for knowledge transfer. "You cannot completely outsource it forever," said Anjul Bhambhri, VP of big data at IBM in a phone interview.

Benefits Of Outsourcing

By far the top benefit of working with an outside organization is speed, said a number of people.

"The pros of outsourcing are clear -- you can get the results faster, and don't have to hire hard-to-find and expensive data scientists," Gregory Piatetsky-Shapiro, an analytics and data mining consultant and editor of KDnuggets, said in an email. Outsourcing "could be entirely appropriate for smaller companies or companies where data is not the main focus or gives them competitive advantage," he said.

A subtler advantage of using an outsider is the chance to examine the data with fresh eyes, without old assumptions or bad habits. "People internally look at the data same way, and it’s sometimes harder to step back, to walk away from history and process and comps," said Brooke Niemiec, divisional VP of customer relationship marketing and loyalty at JCPenney in a phone interview.

And unlike the inside groups, which might be bogged down with ad-hoc requests, an outsider has time for "exploratory analysis," said Niemiec, whose internal team of data analysts numbers around 20 people, including a Ph.D. statistician. "I feel I’m in a fortunate position," she said.

Even so, JCPenney outsources some parts of its data analysis while it works on "operationalizing" data science and driving it through the organization, said Neimiec. The goal is to spend less time reporting and much more time on "interpreting, exploring and communicating" insights from the data, she explained.

Niemiec acknowledged the political realities inside organizations. "You should want to own your major, strategic initiatives," she said about big data and the insights that can flow from it. "But you might also want third-party validation."

Some outsourcing is inevitable because assembling a team entirely in-house is unrealistic, said David Steier, director of information management for business consulting firm Deloitte in a phone interview. "Frankly, a lot of companies have no choice," said Steier, who told InformationWeek previously that data science is a team effort.

These teams will need functional, industry and horizontal skills, as well as technology and interface-design expertise, he said, adding that in conjunction with using outside help, every organization should "find those pockets of hidden talent" -- people with quantitative skills.

A final plus of outsourcing is the opportunity to scale the analysis workload up and down, depending on unique conditions. "Many organizations had huge [marketing] campaigns around the Olympics," said Accenture’s Mulani about a number of large, complicated, time-sensitive marketing analytical projects his company helped out on. Nevertheless, these were one-off projects. "We see that all the time," he said.