Superspeed Analytics

It only makes sense that as business intelligence applications get better at analyzing data, we'll move on to data that's harder to analyze. And that's exactly what's happening.

For a lot of cutting-edge organizations, that means examining unstructured information like stored documents and e-mail messages, or rapidly flowing data like the numbers that flow off a trading floor or the locational information yielded from a supply chain. Such data can be analyzed through the standard BI structure of extraction and loading into a data warehouse. But what if you don't have that much time?

This week we're running a story on two startup companies that are attacking the problem of real-time analytics. As Rick Whiting of InformationWeek explains, RiverGlass Inc. and StreamBase Systems Inc. are building software that can monitor and analyze high-throughput data streams -- the kind that don't yield themselves to slow, deliberate analysis.

RiverGlass software merges data from multiple streams and then both models and analyzes that data in real time. It's designed to be able to detect patterns in information that can help it identify things like investment risks, for example. The technology, expected for release this summer, was developed at the University of Illinois' National Center for Supercomputing Applications. (The University of Illinois also happens to be my alma mater, and to possess, as of this writing, the longest winning streak in men's college basketball. But I digress.)

Like RiverGlass, StreamBase Systems has its roots in the academic world. Its software got its start at the Aurora project, a joint undertaking of Brandeis University, Brown University, and MIT. The application combines an extended version of SQL -- the language for developing database queries -- with a special query engine that analyzes data as it streams by. StreamBase formally debuted this week.

Business intelligence, it seems, keeps on getting more intelligent. That's inevitable, if you think about it. Whatever we decide we can quantify -- and that's a mighty big chunk of the world -- you can pretty much bet we'll eventually decide we also want to analyze, learn from, and use to build advantages. Even when that means learning from data almost as quickly as it's born.