I have yet to see satisfying criteria for assessing social-media analysis tools. Definitive specifications? As Ovum analyst Tony Baer puts it, "that stuff is all around but amorphously defined."
Businesses are spending good money on SMA tools, too often on tools with weak analytics and weak interfaces. I was shocked by one tool I recently saw, both by the tool's shallow sentiment analysis and '90s-vintage analysis capabilities and by the demoing manager's unawareness of same. (Nonetheless, the company recorded $62 million in 2010 revenue, total for all products.)
I suppose the excuse is that social platforms are pretty new, so expectations shouldn't be high. Lame.
Let's raise the bar. I know a worthy challenge when I see one, so here's what I look for in a social analysis tool.
No to Silos, Yes to BI
Start with the thought that most of us are not really interested in analyzing social media, are we? Our real interest is the business value of social content, which lies in messages, people, and networks, not in the platforms themselves.
We're best served by a conception of social networking that transcends any given platform or medium, that bridges into the enterprise systems that mediate and record value-generating ($$$) business transactions.
Yet most SMA tools treat social media in a siloed fashion. They don't recognize the linked mutuality of social and conventional touchpoints -- people post about restaurants they've eaten in, and conversely our purchases are influenced by what we read on social networks. Tools also tend to silo data from each social platform (Facebook, Twitter, blogs, forums, etc.).
As a business analyst, customer support rep, or marketer, I'd want to know not only what was said where online, I'd want also to know who heard the message, whether a message crossed platforms, for instance from a blog or forum posting to Twitter, and what hearers did in response.
Silos mean walls. Down with the walls. Enough with the silos. Understand and account for the networks!
Continue with the admission that while Facebook and Twitter and the like are still young, business-adapted data analysis isn't. Modern BI dates to the late '80s. Every leading BI tool offers not only dashboards and reporting but also dimensional models and pivot tables for interactive, exploratory, visual data analysis. You wouldn't know it by looking at many notable SMA tools (or, by the way, at leading mid-market survey tools). Their best is static dashboards and reports offering limited ability for data filtering and report parameterization. They offer nothing approaching the drag-and-drop, multi-dimensional cross-tabulations that every leading BI tool supports.
The most disappointing example I've seen is heavily-promoted SMA software launched last year by an analytics giant, whose version 3.0 appears still not to benefit from the vendor's deep BI and analytics competence.
What I Look For
SMA silos and weak interfaces are my two big points. Now let's jump down a level. I'll offer a slew of criteria, broken out in six categories. A caveat: Neither the categories nor the criteria are meant to be comprehensive nor deeply explained. I'm looking for gaps, so the criteria I list here relate to elements that are missing from a large proportion of social-media analysis tools now on the market. Any given tool doesn't have to cover every function I'll list -- I'm not a fan of requirements checklists -- but every tool should cover a good selection of the criteria, whichever suit the business goals the tool will support.