My Feelings About Sentiment Analysis

Growth in visibility, acceptance, and adoption of automated sentiment solutions has been unhindered by the debate about automated sentiment analysis. Growth is fueled by solution-provider innovations -- Clarabridge yesterday announced availability of a text/sentiment API, facilitating inclusion of analyses into applications, joining solutions from rivals including Attensity, Expert System, Saplo, Lexalytics, and Open Amplify, the latter three offered additionally as free services -- and by positive customer experiences including the success stories reported by Elizabeth Glagowski in an article, Getting Emotional About Sentiment Analysis, that appeared on June 28 in 1to1 magazine.

Liz interviewed me for her article. With her permission, I will share the full interview with readers.1to1> How does sentiment analysis deepen a company's understanding of customers?

You can tell a lot about customers by analyzing purchases and services usage and inquiries, but transactional records won't tell you the Why behind consumer behavior -- what motivates your customers -- and they won't tell you anything about the folks whom you want to be customers but aren't yet. For a complete picture, companies need to listen to the "voice of the customer," and to get at the richness of the VOC message, they need to do sentiment analysis, to understand attitudes, opinions, and emotions in the customer's (and prospect's) own words.

Whether you chose a manual approach -- good for very focused problems -- or an automated solution, sentiment analysis is key to understanding and acting on the voice of the customer.

1to1> How popular is it in the marketplace today? Are there certain industries using it more than others?

The dozens, even hundreds, of "listening platforms" on the market attest to awareness of the need to at least monitor, if not to also measure and analyze, consumer sentiment. Uptake of more powerful tools, which apply linguistic natural-language processing to the problem, has been strongest for customer-satisfaction initiatives in hospitality and certain segments of consumer goods, but in other industries has lagged problem awareness.

1to1> What sort of new insight is being collected, and how is it being used? Is it being integrated with other customer data or departments?

There's a sort-of sophistication hierarchy, one level building on another: monitoring, measurement, analysis, and intent. Possibilities in the consumer realm range from, for starters, detecting brand or product mentions to, at the high-end, picking up signals that indicate sales, cross-sell, and up-sell opportunities, churn (customer loss) likelihood. The more you know about the customer, the more accurate your predictive models can be.

And if you can link consumer sentiment to purchases, inquiries, and complaints, you can prioritize handling according to total customer value, you can better assess if you really are going to lose a customer over a product issue, you can determine which sales person or staffer needs further training, etc.

1to1> What's the biggest benefit to using sentiment analysis?

Organizations are realizing huge ROI gains in moving to automated sentiment analysis. I know of one large consulting firm that reduced employee-survey analyses from one week's work for five staff to half a day's work for one employee. A survey I ran last year found that organizations were gaining very significant insights, through sentiment analysis, that have led to better product and service design, faster and more effective problem resolution, improved brand image, and boosted ability to understand marketing effectiveness.

1to1> What's the biggest challenge?

Awareness of the benefits, and of the real limitations, of automated sentiment solutions is a challenge, also the need to reengineer entrenched work practices to harness new methods. The biggest challenge, however, may be a rear-guard reaction against automated methods by folks in the "measurement" industry who seem threatened by new, "not invented here" methods.

1to1> How organic is it? Does it need to be managed in real time?

Smart, responsive enterprises have effectively been doing sentiment analysis for years: they've been listening to customers and the market. The natural next step is to automate analyses, to take advantage of computers' speed and power in order to build out and systematize efforts.

Technologies are definitely starting to operate in real-time... and beyond. They can not only analyze and automate response to opportunities and threats as they emerge; via predictive modeling, they can drive pro-active steps that create opportunities and close vulnerabilities.

This said, I'll reemphasize that organizations can work their way up from basic monitoring and engagement to full-blown, predictive analytics at a pace that makes sense given needs and budgets.

1to1> What does the future look like?

In the relative near-term, sentiment analysis will become a routine part of of the search toolkit. We're seeing this already in the form of star ratings for hotels and restaurants that both Google and Bing compute via sentiment analysis of reviews. Expect "search by sentiment" to become common, also for sentiment analysis to be built into a variety of enterprise and line-of-business applications, for CRM (customer relationship management), surveys, warranty and claims management, e-mail and messaging management, and news and social-media analysis.

Look also for feature-level sentiment resolution to become the standard. "Features" may include the names of individuals, companies, and products; they may include topics and general themes. Many tools currently do sentiment analysis only at a document level: a pretty blunt weapon. Finer-grained analysis will lead to better solutions that deliver more accurate and more actionable insights.

A bit beyond, look for sentiment solutions to go "beyond polarity," beyond the positive/negative/neutral classifications now supported to classifications based on emotions (e.g., happy, sad, angry), on intents (likely to buy, undecided, unlikely), and really on any set of outcomes sought by business users.

1to1> What advice do you have for companies looking to add or improve on their sentiment analysis programs?

Start with a modest, solvable business challenge; aim for the "low-hanging fruit" in order to gain experience and build support. This advice of course applies to just about any technology initiative. I'll add that hosted and "as a service" options -- and there are quite a few -- will generally lower entry costs and accelerate time to insight.

Elizabeth Glagowski's article, Getting Emotional About Sentiment Analysis, incorporated material from this interview.

If you'd like to learn more about sentiment-analysis applications and you're in the New York area, do attend an evening New York Technology Council meeting, Wednesday, July 21, 6-8 pm. I'll be speaking and moderating a panel, title Analyze and Optimize Your Organization's Social Media Presence.

Otherwise, I'll continue to monitor and report developments, including initiatives that go beyond polarity, toward the emotions, motivations, and intent signals expressed in human communications, fuel for next-generation sentiment technologies.Growth in visibility, acceptance, and adoption of automated sentiment-analysis solutions has been fueled by vendor innovations and by positive customer experiences, reports Elizabeth Glagowski in 1to1 magazine. Liz interviewed me for her article, and with her permission, I will share the full text of our exchange with readers...