Big Data Sites Filter Election Noise For Facts

What's Obama's position on national defense? Where does Romney stand on healthcare? With all the heated rhetoric flying around these days, including an endless stream of gassy pundits on cable news channels, it's often hard to get a straight answer. But thanks to a new semantic search platform from Atigeo, a big data analytics company, U.S. voters can quickly discover where the candidates and their respective parties stand on key issues.

The two nonpartisan sites, Red2012Blue and Blue2012Red, have identical functionality--the only difference being which political party gets top billing. Naturally, Republicans prefer the former, Democrats the latter.

"My home--and extended family--is a house divided, red and blue. And we were collectively railing at the television about pundits and analysts twisting words," said Atigeo COO and CSO Christopher Burgess in a phone interview with InformationWeek. "We said, 'Wouldn't it be nice if we could just show what the candidates and their parties are saying, and filter out the noise?'"

[ Read about how the NYPD and Microsoft are using big data technology for police work. See NYPD, Microsoft Push Big Data Policing Into Spotlight. ]

Burgess' relatives reminded him that Atigeo's big data semantic search technology is capable of doing just that. Called xPatterns, the platform quickly analyzes a steady stream of unstructured data, such as the billion or so tweets that Twitter users send every few days, and discovers relationships between disparate sources of information, including social media posts, blogs, images, and video and audio clips.

"The xPatterns platform is just that, a platform," said Burgess. It's an application framework for building big data apps, and enterprises can deploy it on-premise or in the cloud.

The Red/Blue and Blue/Red sites ingest feeds from a variety of sources managed by the Obama and Romney camps. This information may include Facebook and Twitter posts, blogs, RSS feeds, and more. "The data is indexed, and concepts are drawn from the data," said Burgess. "It's far more than a semantics search engine or a machine language tool. We reveal the patterns within the data set. It's all about producing information that somebody can act upon."

Voters need not be data scientists to use Red/Blue and Blue/Red. Each home page includes a row of trending political topics, such as "Health Care" and "Immigration Reform." Click "Economy," for instance, and xPatterns lists the most recent economy-related posts from each side. For instance, when we last checked, Republicans were griping about the growth of the national debt under Obama, while Democrats were questioning Romney's tax history and foreclosure policies. You can click a link to read an article, or drag it to a circle in the center of the screen to compare what both parties are saying about the issue.

Atigeo says xPatterns improves its analytical skills over time by studying how a user interacts with structured and unstructured data.

Red/Blue and Blue/Red are designed to demonstrate xPatterns' capabilities, but the sites provide a community service as well. Each includes links to voter registration sites for all 50 states. "Our intent here is to educate the voter, more so than sell our technology," said Burgess. Developers are currently working on other xPatterns applications, including ones for analyzing medical research documents, United States patents, and computer-assisted coding.

"We have one project called PubMed.xPatterns.com. It's a better example of what a commercial implementation of xPatterns might look like. We've digested the entire National Institutes of Health PubMed corpus and have 400,000 articles available," Burgess said.

"Our intent here is to demonstrate that we can deal with a big data set, and that we can display relevance for medical research. That's a powerful connection of multiple databases and revelations of patterns that we believe will jump-start research," he added.

InformationWeek is conducting a survey on big data. Take our InformationWeek 2013 Big Data Survey now. Survey ends Aug. 31.