Do sentiment-analysis tools pass the accuracy test? Here are five tests along with results using freely available products.
Test two is a polysemous word (a word with multiple meanings) where one meaning is a sentiment indicator. Try "kind" with Tweetfeel and you'll see the issue: It uniformly keys on "kind," a sign of positive feelings, where "kind" very often is used in the sense of "type" or "variety."
How do you disambiguate usage to fix this confusion? One basic step is to look at surrounding words. "A kind," "the kind," and "what kind" point to likely "type" or "variety" use.
Test three is sentiment analysis of messages with multiple opinion holders.
Try a search on words such as "said" or, on Twitter, "RT" ("retweet").
Here's an example of a tweet that was, indeed, misclassified:
"RT @ShayIzKilla: Im hating RT @ChocolateWast3d: Oxtails stew on deck.< #Oxtail wait deh where my plate at"
Tough stuff, and I don't mean oxtail, which is tender if you cook it long enough. Here, the original poster implicitly likes oxtail stew given that he or she is about to eat some: "Oxtails stew on deck." The response "Im hating" is negative but not so strongly, the equivalent of "yuck."
That response elicits another, a positive one, "#Oxtail wait deh where my plate at."
Folks, this is how people communicate on social networks; it's "natural language." If you claim to do sentiment analysis, you have to handle it. Send the tweet + RT-response + RT-response to a Twitter sentiment engine.
The freebie toy tools may get tripped up by the language, and regardless, they likely won't distinguish the three opinion holders and their three opinions. They'll give an overall sentiment rating which, whether correct or incorrect, is wrong given that what seems a single message is really three.
"Damn. I could eat jerk & oxtail a few times per wk RT @IMSTAIN:
Had Jerk Chicken Once...#Lowkey Threw Up RT (cont) http://tl.gd/2jra75"
I searched on "oxtail," so I expect the sentiment rating to be for "oxtail," reflecting the text "I could eat jerk & oxtail a few times per wk" with the intensifier "Damn." (I added the poster's name to the search, in order to bring up the particular message I was looking for, only after observing the incorrect classification.) Instead, socialmention either incorrectly keyed on "Damn" as a sign of negative sentiment -- although any tool that analyzes slang-filled social media should know better -- or is fooled by the "Threw Up" associated with jerk chicken.
The key to passing tests three and four is the ability to break messages into appropriate chunks, which may be phrases, quoted strings, or retweets within a longer message. For example, the folks from Conversition say that "Tweetfeel is meant to measure a subset of data around nouns," which I take to mean that it does focus sentiment analyses on searched-on terms.
The Agile ArchiveWhen it comes to managing data, don’t look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyIT’s tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.
Join InformationWeek’s Lorna Garey and Mike Healey, president of Yeoman Technology Group, an engineering and research firm focused on maximizing technology investments, to discuss the right way to go digital.