The ACM Looks at Sentiment Analysis

Our Sentiments, Exactly in the April issue of the Communications of the ACM tackles sentiment analysis. The subhead: "With sentiment analysis algorithms, companies can identify and assess the wide variety of opinions found online and create computational models of human opinion." (I suppose that last bit, the geek-speak, suits the audience. The Association for Computing Machinery is, after all, essentially an industry association for computer scientists.)

Author Alex Wright interviewed me for the article, and with his permission, I'll share our conversation...Alex> "Sentiment analysis" seems to be an emerging term that doesn't yet have a clear definition. How would you define sentiment analysis?

Seth> Sentiment analysis is a set of algorithms and tools for identifying and extracting a) features that express attitudes or opinions, b) attributes that indicate sentiment polarity, intensity, and other characteristics, and c) the topics those sentiments and attributes apply to. Sentiment analysis may further include aggregating sentiment across sources, documents, or document sections and studying trends and correlation of sentiment with events and demographic information.

Alex> What are some of the key technical challenges involved in sentiment analysis vs. more traditional textual analysis?

Seth> Sentiments are very different from conventional facts. Cultural and contextual factors come into play: who's "speaking," where is that person speaking, who is he or she speaking to? There's lots of slang -- phat, cool, bomb, snap -- and ambiguity -- "sinful" is a good thing when applied to chocolate cake -- as well as sarcasm, irony, and idiom. You don't see "Genistein inhibits protein histidine kinase... Not!" in a scientific paper.

Alex> What do you see as the major market opportunities for sentiment analysis software? Who's using it successfully today?

Seth> The leading app today is in the Voice of the Customer-Enterprise Feedback Management-Customer Experience management space: efforts to gather and assess opinions about products and services from customers, prospects, and broad consumer markets. The basic aims are to improve product and service quality, customer support, brand and reputation protection, marketing.

Sentiment analysis has had notable successes in the hospitality and consumer products arenas and in the form of recommendation engines.

Alex> Who are the major software vendors in this space? I'm already talking to Infonic [whose text technologies have now been transferred to Lexalytics] - who else comes to mind?

Seth> There are many companies in this arena, the majority of them small (like Infonic/Lexalytics) or start-ups. The sophistication of capabilities varies widely. Among the text-analytics companies, you could check out Attensity, Clarabridge, Jodange, SPSS, Temis. There are also services such as Biz360, Nielsen BuzzMetrics, and TNS Cymfony. SentiMetrix is a typical start-up. These are the ones that come to mind.

Alex> Who would be the most likely consumers of sentiment analysis data inside a company -- market researchers? Business Intelligence people? Who else?

Seth> Sentiment data is often presented in BI dashboards and other familiar interfaces, but sentiment (and general text) analysis is not yet broadly done in conjunction with conventional BI, with analysis of numerical data drawn from transactional and operational systems. Unified analysis is coming, but it's not here yet.

For now, users are typically market researchers, business analysts, customer-support staff, etc. Hosted services are very popular because in-house IT capabilities hasn't kept up with the demand.

There was a bit more. For instance, I suggested that Alex contact Bo Pang and Lillian Lee, authors of a definitive monograph, Opinion mining and sentiment analysis, and also Professor Bing Liu, but the above was our core interview.

I hope this additional material was useful. I'll actually be speaking on this topic at the May 12-13 Enterprise Search Summit in New York, and we're fortunate to have Bing Liu presenting on it at this year's Text Analytics Summit, June 1-2 in Boston."Our Sentiments, Exactly" in the April issue of the Communications of the ACM tackles sentiment analysis. The subhead: "With sentiment analysis algorithms, companies can identify and assess the wide variety of opinions found online and create computational models of human opinion." Author Alex Wright interviewed me for the article, and with his permission, I'll share our conversation...