Big Data's Surprising Uses: From Lady Gaga To CIA

(click image for larger view and for slideshow)

If you're a regular reader of InformationWeek's big data section, you're well aware that data scientists are hard to find. And apparently easy-to-use data science tools are a challenge to find as well.

According to Stephen Purpura, CEO and cofounder of Context Relevant, a Seattle-based big data company, today's development tools for data scientists are largely "medieval" -- a problem his year-old startup hopes to correct.

Context Relevant provides hosted and on-premises data analytics software. Its apps run out-of-the-box on top of Hadoop and other MapReduce engines for business tasks that include fraud detection, click prediction and analytics for Splunk's big data search and monitoring software.

Context Relevant may be new, but its management team has plenty of big data experience. For instance, the company's VP of engineering, Jim Walsh, led development of Microsoft's Cosmos distributed data storage and analysis environment, which serves as the basis for many Microsoft products, including Bing.

"We focus on predictive analytic applications because (our) team is composed mostly of machine learning experts. That's the kind of value we can add," said Purpura in a phone interview with InformationWeek.

[ You've set up your big data appliance and figured out how it works. Now what? Learn How To Beat The Big Data Disconnect. ]

Of course, Context Relevant isn't the only vendor providing predictive analytics apps, and the startup has formidable competitors such as Opera Solutions, Palantir Technologies, as well as tech giants IBM and SAP.

"Our base solution will run on anything from a laptop to a big, 1,000-node Hadoop cluster, Purpura said.

Fine, but how does Context Relevant distinguish itself in such a crowded field?

"We're using machine learning to make horizontal applications that auto-configure themselves to the data," said Purpura, adding that many of his competitors focus instead on "vertical-specific" apps.

Customers can use Context Relevant source code to quickly build big data apps.

"The source code, which runs on top of our libraries that run on Hadoop, is very small. It's typically a hundred lines or less of Python code. It's really easy to understand," Purpura said.

For one customer, Context Relevant's approach has allowed it to cut development time for its predictive analytics-based apps from months to days, Purpura claimed.

"Even people who don't know a lot about data science can read through (the source code) and get a good understanding of what's happening," he said. "They can use this understanding to riff on it a little bit, to produce new applications that are similar in style, but more tailored to what they want to do."

Fraud detection is another area where Context Relevant's technology shows promise.

"Once you have the capability to scan through lots of data, you develop enough baselines about how people are performing to make recommendations," Purpura said.

"Finding anomalies with the way people are behaving is actually pretty easy," he said. "And our system can do it in near real time, which is much faster than existing systems on the market."

Context Relevant may be a newcomer to the analytics field, but it already has paying customers, Purpura said. Its highest-profile client thus far is Concur Technologies, a travel and expense management provider, which uses Context Relevant's products to help its corporate clients find spending anomalies, reduce expenses and so on.

Purpura said other large enterprise clients haven't granted Context Relevant permission to release their names, but added that "more announcements would be forthcoming."

"There are very few 11-month-old startups with C-stage funding that have real customers paying them big dollar amounts to deliver solutions," he added. "The need and demand in this market is so hot."

Attend Interop Las Vegas, May 6-10, and attend the most thorough training on Apple Deployment at the NEW Mac & iOS IT Conference. Use Priority Code DIPR02 by March 2 to save up to $500 off the price of Conference Passes. Join us in Las Vegas for access to 125+ workshops and conference classes, 350+ exhibiting companies, and the latest technology. Register for Interop today!

Federal agencies must eliminate 800 data centers over the next five years. Find how they plan to do it in the new all-digital issue of InformationWeek Government. Download it now (registration required).