Twitter Helps Build Social Data Ecosystem

It almost goes without saying, that good, clean, comprehensive social data is central for any meaningful social-media analytics initiative. Note that I wrote "almost."

Judging from market buzz, much of the social-intelligence audience is swayed by appearances--like slick dashboards--while paying little attention to the value and reliability of the analyses and insights delivered. Social enterprises and social networkers need to rethink priorities. We need social data quality and trustworthy analytics.

There are a number of musts for quality social data. Number one is completeness and also control: You need to be able to filter the "fire hose" for just the data you need to be able to select according to characteristics such as location, content (keywords, topics, and hashtags), and perhaps for selected individuals (by their social handles). You may also need access to a repository of past postings, dating back months or even years.

[ Want more on must-haves in sentiment analysis? Read 10 Tips: Tap Consumer Sentiment On Social Networks. ]

The second must is context, essentially the metadata envelope that describes the social content. Number three is timeliness. Not everyone needs a real-time feed, but consider the business problem. You need instantaneous alerts if you're engaged in automated financial trading, for example. A modest latency is tolerable for reputation management and customer engagement, and a delay of a few days might suffice for market research.

You also need analytical tools that will help you make proper and profitable use of the data you collect. Enter Twitter, whose value to business users depends on a quality social data ecosystem.

Twitter-Certified Products

The social-data challenge surely spurred the recent launch of the Twitter Certified Products Program. It's really a step to promote, for the benefit of Twitter data consumers, an ecosystem of high-grade supporting products and services. I hope that Facebook, LinkedIn, and other social platforms, which have business and technical architectures different from Twitter's, find ways to follow suit!

Two of Twitter's three partner categories--analytics and data reseller--respond to the good, clean, comprehensive social data need. Products in these categories are foundational; social customer engagement based on finding and classifying postings doesn't scale without them. That's why selling point number one for social-engagement leader Radian6 is capture of the entire social web, "hundreds of millions of posts each day, including Facebook, blogs, news sites, discussion boards, video, and image sharing sites." It's also why Radian6 provides quite a number of analytical options, via plug-in capabilities from third-party providers.

Data Harvesting, Analytics, And People

Comprehensive social-data harvesting (including Facebook, Twitter, LinkedIn, blogs and the rest) is beyond the budget and ability of most organizations, creating market space for data resellers. Twitter has certified three data resellers, DataSift, Gnip, and Topsy, and the first two cover the spectrum of social sources.

Similarly, advanced social-data analysis is beyond the build-it-yourself interests or capabilities of most companies. Social content analysis, for instance, applies natural-language processing (NLP) processing algorithms to identify entities, relationships, and sentiment in social and online text, and it requires specialized semantic-analysis software. DataSift, Gnip, and Radian6 partner with third-party text and sentiment and social-network analysis providers, or you can pull data from DataSift or Gnip into an analysis system you've built yourself. (Full disclosure: DataSift, Gnip, Topsy, and Twitter engagement-and-analysis partner Attensity are sponsors of the Sentiment Analysis Symposium, a conference I organize, slated for October 30, 2012 in San Francisco.)

Even the best analytics and the most comprehensive data can only suggest insights, however. There's no substitute for human judgment in selecting data, designing analysis processes, training the tools (to accommodate vocabularies and expressions unique to particular business domains and functions, for instance), verifying results, and applying them appropriately. (Kohlben Vodden provides valuable guidance in his article, "Social Data: Stop Listening and Start Thinking.")

Listening, analyzing, automating: These are matters of importance in marketing, market research, customer experience, media, government, and public affairs and to technologists who support these areas. Focus on leading-edge social intelligence rooted in what's practical today. Think about how and where you can learn, from experts and peers.

Beyond Sentiment Analysis, Social

Sentiment is a key social-intelligence element, whether you're looking only at polarity (positive/negative/neutral classifications) or more expansively, at extraction of emotion (happy, sad, angry) and intent (to buy, to cancel, or to vote). Our interest is, more broadly, in signals and sense-making, meaning in patterns that can be detected in single sources and across multiple data sources, and in situational analyses that account for context in guiding decision making.

Considerations go beyond social, extending to integration of social-derived information with enterprise data. When it comes to data-driven marketing challenges, social media interaction is probably more telling of your relationship with your customers than traditional interactions, observes Trillium Software's Michele Goetz in the article "Does Data Quality Matter in Social Media?" So quality means pulling data from all available sources. We're after a big and broad data picture, applied to market research, customer experience, financial markets, media analysis, public affairs, you name it. We're talking new-era business intelligence and operations.

Correct, social-infused business decision-making in high-velocity, high-volume data environments demands careful attention to the quality of the data and the quality of the analysis. The Twitter certification program, and any other opportunity to showcase exceptional providers and sound expert and peer guidance, takes us a long way toward our social intelligence goals.

Seth Grimes is an analytics strategist with Washington DC-based Alta Plana. He chairs the Sentiment Analysis Symposium, which is slated for October 30, 2012 in San Francisco.