Five Ways To Fool A Twitter Sentiment Tool - InformationWeek
IoT
IoT
Software // Information Management
Commentary
12/16/2010
01:54 PM
Seth Grimes
Seth Grimes
Commentary
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Five Ways To Fool A Twitter Sentiment Tool

Do sentiment-analysis tools pass the accuracy test? Here are five tests along with results using freely available products.

Like many social-media analyses, the idea for this one originated on a social platform. In this case it was Twitter, the most accessible, easy, and free-wheeling of them all and hence a great place to exchange information and opinions.

Tweets (and other social and online postings) have immense business value, in particular as a "voice of the customer" information source about products and services and also about politics, family, and just about every other aspect of our daily lives. Their subjectivity -- they voice opinions and not just facts -- is what makes them particularly valuable.

How do we get at this business value, at sentiment especially, and how do we make sure we're doing it reasonably well? You do need software to get a complete picture of the online universe. There are dozens of tools on the market, some very good, some not so strong. Accuracy, as measured by precision, recall, and relevance, is essential. I'll suggest a few simple tests that can help you assess the precision (proportion of cases correctly classified) of tools you may be considering, focusing on Twitter sentiment analysis.

One of my tweeple, @Pythonner (a.k.a. David Nadeau) posted a lookout, Twitter + sentiment analysis, nicely integrated: http://www.tweetfeel.com/. The Tweetfeel tool from Conversition does make it easy to view Twitter-message sentiment via what the company characterizes as "real-time Twitter search with feelings using insanely complex sentiment analysis."

Accurate automated sentiment analysis is insanely difficult given the complexity of human language and expression. As expert systems pioneer Edward A. Feigenbaum observed, "Reading from text in general is a hard problem, because it involves all of common sense knowledge."

Text is full of subjectivity. There is no text-analysis problem harder than correctly parsing attitudes, opinions, feelings and emotions.

Five Tests

My first test is sentiment classification for someone who has recently died. Tools trip easily on that one. Tweetfeel is no exception, as seen in a search on Elizabeth Edwards.

The tweet, "ummm I feel totally @#$%!... elizabeth edwards lost her battle with cancer??? omg!!!!! how sad" is NOT, as rated by Tweetfeel, negative.

The folks from Conversition did make clear, "TweetFeel is intended as a quick, fun tool. It's a 'give back to the community' application."

Prompted by another of my tweeple, @yehaskel (a.k.a. David Yehaskel), I'll offer four more tests, essentially quick and easy ways to trip up (most) Twitter sentiment tools but also good starters for any evaluation of the more capable tools on the market, albeit not available for open trial.

Previous
1 of 3
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of the Cloud Report
As the use of public cloud becomes a given, IT leaders must navigate the transition and advocate for management tools or architectures that allow them to realize the benefits they seek. Download this report to explore the issues and how to best leverage the cloud moving forward.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of November 6, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll