Tweets Tell Whether You Have A Job - InformationWeek
IoT
IoT
Government // Big Data Analytics
News
11/20/2014
03:36 PM
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Tweets Tell Whether You Have A Job

Twitter data mining could provide governments with an alternative means of measuring unemployment, researchers say.

 8 Doomsday Predictions From Yesterday And Today
8 Doomsday Predictions From Yesterday And Today
(Click image for larger view and slideshow.)

Researchers at universities in Australia, Spain, and the US, in conjunction with UNICEF, have found that Twitter posts can be mined to measure unemployment.

In a study published through Cornell's arXiv.org, researchers Alejandro Llorente, Manuel Garcia-Herranz, Manuel Cebrian, and Esteban Moro "demonstrate that behavioral features related to unemployment can be recovered from the digital exhaust left by the microblogging network Twitter."

"Digital exhaust" is a curious choice of words, because it implies that social media data is a worthless byproduct of online interaction, something to be cast aside. Yet the researchers' findings suggest the very opposite: Social media exhaust is the primary product. It's gold, rather than garbage, and social media users don't realize the value they're throwing away. The social media realm is a charity to benefit businesses.

Three decades ago, the technologist and writer Stewart Brand famously observed the tension inherent in how we value information.

[Does big data need a social media approach? Read Does Big Data Need A 'LinkedIn For Analytics'?]

"Information wants to be free," Brand said in one of his several variations on this theme. "Information also wants to be expensive. Information wants to be free because it has become so cheap to distribute, copy, and recombine -- too cheap to meter. It wants to be expensive because it can be immeasurably valuable to the recipient. That tension will not go away."

Facebook, Google, Twitter, and the rest of the social media and advertising industry want people to believe that their online work -- their posts, pictures, and associated data -- isn't worth anything, in order to capture the full value of this free bounty of insight. Pollute the world with your digital exhaust; we'll clean up, all the way to the bank.

Llorente and his colleagues used a data set of 19.6 million geolocated Twitter messages in Spain from Nov. 29, 2012, to June 30, 2013, and a data set detailing unemployment across various regions of the country to uncover a relationship between economic metrics and social behavior. Interestingly, they noted a correlation between misspellings in tweets, which they take as a proxy for education level, and unemployment.

(Image: Wikimedia)
(Image: Wikimedia)

The researchers consider their success in using Twitter posts to assess employment status to be a "a proof of concept for how a wide range of behavioral features linked to socioeconomic behavior can be inferred from the digital traces that are left by publicly available social media." And they argue that governments might be able to adapt social media surveillance as an alternative to more costly traditional data gathering methods related to public policy.

"The immediacy of social media may also allow governments to better measure and understand the effect of policies, social changes, natural or man-made disasters in the economical status of cities in almost real-time," the researchers state.

Others have already reached this conclusion and are monitoring social media for threats to the powers that be, at home and abroad. Intelligence agencies have been wise to the value of social media exhaust for years, both for the overtly expressed sentiment and for the web of personal relationships exposed through social network links.

In fact, data mining to assess aspects of society has become commonplace. Google began using search queries to assess flu infections in 2008. This year, researchers have demonstrated that Twitter posts can be used to infer whether the person posting is ill. And web pages are laden with tracking scripts to gather data.

Interpreting that data correctly, however, remains a challenge. Google's flu tracking system turned out to be inaccurate. Worse, it would not be difficult to construct a social media study to support a predetermined conclusion for political or economic gain. So Twitter used as a way to measure employment should be assessed with caution.

Twitter has long been wise to the value of its data. In its IPO filing, it disclosed that it had made $47.5 million selling data to other companies in 2012, up from $28.6 million the year before.

Companies want information to be free, so they can sell it at great cost.

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.

Thomas Claburn has been writing about business and technology since 1996, for publications such as New Architect, PC Computing, InformationWeek, Salon, Wired, and Ziff Davis Smart Business. Before that, he worked in film and television, having earned a not particularly useful ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
asksqn
50%
50%
asksqn,
User Rank: Ninja
11/28/2014 | 5:24:29 PM
Sad
It's a sad commentary when social media is a much more accurate gauge of Main Street, USA than the bloated, corrupt bodies collecting paychecks for being federal employees at the BLS dot gov.
jastroff
50%
50%
jastroff,
User Rank: Ninja
11/21/2014 | 10:47:16 AM
Socio-Political Use of Big Data
Here's a quick summary of how the study worked and what they found:

>> Studied Variables: diurnal rhythm, mobility patterns, and communication styles

>> Data Set: we examine a country-scale publicly articulated social media dataset, where we quantify individual behavioral features from over 145 million geo-located messages distributed among more than 340 different Spanish economic regions, inferred by computing communities of cohesive mobility fluxes

>> Results: Our results show that cost-effective economical indicators can be built based on publicly-available social media datasets.

Some academics had an interesting idea to analyze twitter utterances by these variables, by region, according to formulas. I suspect it won't wipe out the Conference Board job numbers any time soon, unless Spain uses this data to solve its unemployment problem within the year.

But these kinds of studies can be refined, and replicated so they are proven to be accurate, and only then they can give us reliable data. Otherwise, they don't work and don't add to what we know, and don't contribute to policy and what problems we can solve. They are academic exercises for tenure or dissertations.

These researchers are a long way off from proving anything, but time will bring forth more of these studies. Harnessing big data for policy making is a good idea.
jgherbert
50%
50%
jgherbert,
User Rank: Ninja
11/20/2014 | 11:45:50 PM
Re: "Digital Exhaust" is being misused
@SachinEE> "because "wryteeng lyk dis" does not mean one does not have proper education and employment"

 

It may not be 100% accurate but as this is the Internet and we're all entitled to make quick sweeping judgments on other people, so let's assume that anybody writing like that either does not have English as a first language, or has poor education and - by implication - perhaps a lack of employment.

 

Ok, I'm kidding, but some people truly believe that Tweets are an insight into the soul. The Samaritans in the UK recently launched the "Samaritans Radar" app, which monitored Twitter feeds of subscribers' friends and notified the subscriber if they believed their friend was sending tweets indicative of an impending suicide. The Samaritans have since suspended the service after many people and organizations expressed deep concerns about the intent and operation of the app (and perhaps rightly so).
SachinEE
50%
50%
SachinEE,
User Rank: Ninja
11/20/2014 | 10:30:53 PM
Re: "Digital Exhaust" is being misused
"I think almost all can be digged up from digital traces. Simply, people will be revealing their woes in social media to garner sympathy. In anonymous user profile, people can try to act 'cool', in friend circle, everyone knows who you are, how much you earn, whether you had a daownfall -so people tend to be straight froward. Though, there are some 'show off' but it can be inferred, however."

@Zaious: I recently saw a junior of mine post on facebook that he has got a job in one of the core software companies with a six figure annual salary and all I could think of is "wow this would make for a great story for the data mining software to know the demographic of the region from where you belong". Data mining is really hitting gold with social media digital exhaust.
SachinEE
50%
50%
SachinEE,
User Rank: Ninja
11/20/2014 | 10:27:00 PM
Re: "Digital Exhaust" is being misused
"Althought the findings sound promising, individuals can post inacurate information.  What would happen to the other segments of population which do not use social media.  Should we assume that twitter post are a true reflection of the population of study? I think twitter should help those that do not or are searching for a job instead of the opposite."

Twitter posts should not be the base of this study because "wryteeng lyk dis" does not mean one does not have proper education and employment. What the analytic engines do instead, is thrust the data to some data analyzing algorithm to get the most probable result.
PedroGonzales
50%
50%
PedroGonzales,
User Rank: Ninja
11/20/2014 | 9:43:37 PM
Re: "Digital Exhaust" is being misused
Althought the findings sound promising, individuals can post inacurate information.  What would happen to the other segments of population which do not use social media.  Should we assume that twitter post are a true reflection of the population of study? I think twitter should help those that do not or are searching for a job instead of the opposite.
zaious
50%
50%
zaious,
User Rank: Ninja
11/20/2014 | 8:39:55 PM
Re: "Digital Exhaust" is being misused
As the article says, "a proof of concept for how a wide range of behavioral features linked to socioeconomic behavior can be inferred from the digital traces that are left by publicly available social media."

I think almost all can be digged up from digital traces. Simply, people will be revealing their woes in social media to garner sympathy. In anonymous user profile, people can try to act 'cool', in friend circle, everyone knows who you are, how much you earn, whether you had a daownfall -so people tend to be straight froward. Though, there are some 'show off' but it can be inferred, however.
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
11/20/2014 | 4:22:00 PM
"Digital Exhaust" is being misused
From what's described, it sounds like they're misusing the phrase "digital exhaust." Twitter tweets are human expressions of information, opinion, and insight. It's Twitter's social network, and to my knowledge it has never treated tweets as "exhaust." That data is their life blood, and they've made APIs to that data widely available. Aggregators capture that data and offer it as well.

The term "digital exhaust" was coined to describe the data that most enterprises used to routinely throw away. It was described as exhaust because it was not seen as being "business data." Log files, for example, contain super high-scale machine data. IT might have kept this data on a short-term basis as a way to monitor the functioning of IT systems.

Now that Hadoop and other high-scale platforms are available, it's possible, economically speaking, to store this "exhaust data" and analytics firms such as Splunk and others have found ways to correlate this data with business events and customer data to develop business insights. To sum up, Tweets aren't exhaust data. That term describes types of data that used to thrown away because there was no perceived value and there was no economically sensible way to store that information.
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
Digital Transformation Myths & Truths
Transformation is on every IT organization's to-do list, but effectively transforming IT means a major shift in technology as well as business models and culture. In this IT Trend Report, we examine some of the misconceptions of digital transformation and look at steps you can take to succeed technically and culturally.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll