Text Analytics: What It Is, Why You Need It

Gartner defines text analytics as "the process of deriving information from text sources."

Text analytics are used for trying to find the key content across a larger body of information, analyzing the sentiments of individual writers, or classifying information into various topical areas.

To derive business insights from plain text, analytical software uses a combination of natural language processing in which the program actually deciphers and understands the human language that is being spoken; speech tagging in which the various words contained within a text are tagged based upon the part of speech they are; syntactic parsing in which strings of letters are analyzed and grouped into word phrase structures; and entity recognition, which identifies the names of persons, organizations, locations, etc. These text analytics programs can also use artificial intelligence to analyze sets of information and deduce a subjective expression or sentiment behind the words.

"Text analytics can help businesses listen to the right stories by extracting insights from free text written by or about customers, combining it with existing feedback data, and identifying patterns and trends," Terry Lawlor, a product manager at Confirmit, which provides solutions that focus on understanding the customer experience, wrote in a post for TDWI. However, Lawlor also acknowledges what others have noted in the industry -- that text analytics is still in early stages of adoption at most companies.

One reason adoption has been slow is that companies are still getting their arms around how to best plumb the depths of their unstructured data, including voice and text. Second, businesses have to develop internal competencies or find knowledgeable business partners so they can learn how to exploit text analytics to best advantage.

What are the use cases where text analytics is showing promise? Legal eDiscovery is one area where text analytics is achieving significant traction.

"In civil litigation, both sides of the dispute have the obligation to provide documentation," said John Tredennick, an attorney and CEO of Catalyst Repository Systems, a cloud-based eDiscovery service. "Years ago as a trial lawyer, when I first got into the discovery process of a civil litigation, we were looking at document populations of perhaps 30,000 documents. However, with the growth of digital documentation, in a major litigation this document population could expand to 20 million or 30 million documents...Now we use text analysis and predictive analytics in the document review process. By using this process, we often find that 75% to 80% of the relevance in a litigation can be found in a population of the most 6,000 highly ranked documents for relevance...When you're talking about an average cost of $2 per document for a manual review and you have 1.5 million documents to review, this can save companies a lot of money."

Libraries, universities and research institutes use text analysis to research trends and help explore the depths of vast repositories of data.

In one case, the U.S. Army Combat Readiness/Safety Center collected information on the circumstances of military vehicle crashes so it could determine if vehicle technology could have prevented the crashes. It used text analytics tools to review the text of 3,944 military vehicle crash narratives.

How do you decide if text analytics is for you?

1. You are a text-intensive business

If your business is life sciences, library research, marketing, sales, customer service, media or legal research, an analytics tool that helps automate information searches and narrows down the amount of text you have to look at can really help.

2. You want to understand how your customers feel

If your company wants to understand customer sentiment during sales and service, along with what customers are saying about you in social media tweets and comments, text analytics can give you this visibility.

3. You have the ability to start small

Starting any new phase of analytics is best done on small scale where you can see if you company is going to get the value out of the text analytics that you think it will. If you and your end users don't see a value, it's relatively easy to pull the plug. If the analytics work out great for the business, you can expand your work into new areas.