Data Scientist: The Sexiest Job No One Has

I've spent most of my career working with new technology, most recently helping companies make sense of mountains of incoming data. This means, as I like to tell people, that I have the sexiest job in the 21st century. That's not me talking. SmartData Collective, an online community moderated by Social Media Today, has called big data scientist "the sexiest job of the 21st century." Who am I to disagree?

Besides making folks like me feel good, that designation might offer another benefit. It'll help draw new talent into our field. This is currently a big problem, and it's getting bigger every day.

The role of data scientist has changed dramatically. Data used to reside on the fringes of the operation. It was usually important but seldom vital -- a dreary task reserved for the geekiest of the geeks. It supported every function but never seemed to lead them. Even the executives who respected it never quite absorbed it.

[ Small and midsized businesses don't need an on-staff data scientist. Read more: Are You Recruiting A Data Scientist, Or Unicorn? ]

But not anymore. Today the role set aside for collating, sorting, and annotating data -- a secondary responsibility in most environments -- has moved to the forefront. In industries ranging from marketing to financial services to telecommunications, the data scientists of today don't just crunch numbers. They view the universe as one large data set, and they decipher relationships in that mass of information. The analytics they develop are then used to guide decisions, predict outcomes, and develop a quantitative ROI.

Data used to be the backup for instinct and intuition, but now it's instrumental in how companies know which marketing campaign is working, which new markets have the best potential, and which consumers represent the best prospects for growth. Data is the ultimate roadmap, and job candidates who can navigate it are poised for success.

But here's the best part: Since it's not a one-dimensional discipline, data scientists can emerge from just about any field. A good data scientist is someone who has the right tools (math, programming, critical thinking), is self-sufficient (doesn't need someone else to implement his or her ideas) and has an interest in understanding the context in which the skills can be applied. This is what the marketplace seeks.

A marine biologist with a PhD might have a strong grasp of both quantitative and qualitative research methodologies, and those skills perfectly meet the needs of many companies looking to mine data for intelligence they can use to take action. Baseball statisticians can leverage their passion and apply it to solve problems in corporate America. Many of these professionals probably don't even think of themselves as data scientists per se, but they should. There's a huge need, and here's why.

The all-purpose big data market is booming. Many news stories beat the drum of chronic unemployment, but Gartner estimates that there will be 4.4 million IT jobs created to support data analysis in just the next two years. Almost half these jobs will be in the US, and each will lead to three more. This field alone could help turn the job market around.

However, it's clearly not happening now. Countering the good news, Gartner also sees a staggering data scientist shortfall in the pipeline. There are only enough qualified applicants to fill a third of those positions. In other words, despite the importance of the data scientist position, they are set to become a "scarce, valuable commodity."

This is why decision makers in corporate America -- in particular the technology and business-centric professionals reading this -- need to refocus their attention on the institutions they left behind so long ago: middle and high school. That's where the problem begins, and it has to do with a simple acronym: STEM (science, technology, engineering, and math). The private sector needs to fund initiatives that bring young STEM prospects into the workplace to show them how the real world uses the information that people generate and demonstrate how to apply their interests to this discipline -- for instance, data visualization.

Deficiencies in school today will have a crippling effect on the workplace tomorrow, and that's why this is so important. The so-called sexiest job of the century isn't going away soon. Data scientists will lead the way in innovation and economic growth, but we have to help develop the next generation.

IT groups need data analytics software that's visual and accessible. Vendors are getting the message. Also in the State Of Analytics issue of InformationWeek: SAP CEO envisions a younger, greener, cloudier company (free registration required).