NLP, Machine Learning: Killer Combo for Unstructured Data

If you are looking for the next emerging job title or sought-after skill set that will make companies compete to hire you and pay you a premium, consider gaining some expertise in working with unstructured data and natural language processing.

While both machine learning and natural language processing skills are in high demand in the market today, NLP experts are harder to find.

"NLP, even among analytics, is among the hardest skills in the market," Sameer Gupta told InformationWeek. He is the AI and advanced analytics leader at EY, specializing in the financial services market. Gupta also recently authored a whitepaper about using NLP and ML to harness the insights from unstructured data.

It is the combination of NLP and machine learning that will enable organizations to gain insights from unstructured data such as emails, chat transcripts, outbound marketing materials, internal memos, legal documents, and complaint logs, in a way that hasn't been previously possible. Organizations can mine this type of unstructured data for several purposes, including regulatory compliance and sentiment analysis, according to Gupta. While organizations may have gained momentum in efforts to apply machine learning technologies to their structured data, they are still just getting started with NLP, unstructured data, and technologies such as sentiment analysis, he said. That's why there's a high demand for these skills that are still in short supply.

"Some functional and working knowledge of NLP and machine learning is one of the most sought-after skills," Gupta said. He works with companies in the financial services industry, which is a bit ahead of many other industries in terms of applying these emerging technologies but is not as advanced as some of the tech leaders in the space such as Amazon and Facebook.

"Organizations don't know where to start, but that hasn't stopped them from trying or starting," Gupta told InformationWeek. "They recognize there is a lot of insight buried in unstructured data."

For instance, some banks have been around for more than 200 years, and they are competing with digital native startups. Providing superior customer responsiveness is crucial to being competitive against these new market entries. These companies need to be able to do a better job of harvesting data from previous interactions to provide service and messaging that is in sync with what customers want.

NLP and unstructured data can also drive down the costs of regulartory compliance.

"Some of the clients we serve have more than 1,000 people just listening to calls and reading notes and chat transcriptions to be able to see if the interaction meets compliance needs," Gupta said. By automating those processes with NLP and machine learning, organizations can lower the costs of that monitoring.

The NLP market right now, like many new markets, is primarily populated with a lot of startup companies. You won't necessarily find your longtime enterprise software vendor offering this kind of functionality built into their existing products. Yet.

"The days where people could rely on one vendor to provide all the components in the stack are not quite here anymore," Gupta said. "The bigger vendors are responding in two ways -- closing the gap in their overall stack and also making their stack more open and flexible to respond to niche providers and tools.

For companies looking to hire these types of skills in-house, Gupta said there are three levels of skills needed for someone to be an effective contributor. At the base level, there are the modeling skills required -- generally the ability to program in Python. The next level is some knowledge of about a half dozen key NLP techniques. The final level is the ability to combine those first two skill sets and apply them to the business problem. That third level is the key to a successful job candidate, according to Gupta. Having a programming language on your resume doesn't mean you have the experience to know how to apply it in a real-world setting.

Yet hiring the right skills is just one part of a larger effort, Gupta said.

"Scaling all of this in financial services is not just driven by talent," he said. "It's also driven by the right architecture and organization constructs."

To scale it, you must identify the right architecture that enables you to reuse models and efforts.

"The ones who do that will be the most successful."

For more on AI, skills, and careers, read these articles:

Automation, AI Impact: The Future of Work

Data Hunter: The New Sexy Technology Job

AI, Automation Job Trend to Reverse in 2020

Prepping the Enterprise for the AI Apocalypse

Data Science: A Guide to Careers and Team Building

AI & Machine Learning: An Enterprise Guide