Business Intelligence 2.0: Simpler, More Accessible, Inevitable

Why do three billion people a day use a Web site that has a Spartan interface, a weird name (Google) and the capacity to do only one thing (the original search engine, that is, not the newer applications)? Perhaps the reason is that it always gives you more than you asked for, but what you want is usually on the first page. No one needs to install software, to do upgrades or, for that matter, to pay for it. And no one needs a training class.

Contrast this with Business Intelligence (BI) tools today, with complicated interfaces, software that is very expensive to purchase and maintain, and version upgrades that are often painful. These tools generally offer very broad functionality, but somehow it's never quite enough to solve the problem. There are, at best, a few million copies licensed worldwide (across all BI vendors), yet more than a third of these seats are shelfware, according Optima Publishing's OLAP Survey 4.

One could argue that phenomenon of the Web is so unique that it is unfair to measure BI's performance against it. The problem is that this is the second time BI has been eclipsed by another technology, as proven by the fact that there are more than 200 million licensed copies of Microsoft Excel in use today.

Rest assured, the current era of BI is coming to an end and will be succeeded by a BI 2.0 era that promises simplicity, universal access, real-time insight, collaboration, operational intelligence, connected services and a level of information abstraction that supports far greater agility and speed of analysis. The motivation for this "version upgrade" for BI is the need to move analytical intelligence into operations and to shrink the gap between analysis and action.

BI 1.0 Defined

Despite its drawbacks, BI is not a failure. In most ways, the facility for getting information to people where and when they need it is dramatically better than it was more than a decade ago, when reports were usually paper-based and required months of development to line up the data. But if you boil BI down to its basics, it is derived from only two things: data and reports. Most of the effort in BI in the last decade has been focused on the data issue data integration, data quality, data cleansing, data warehouse, data mart, data modeling, data governance, data stewardship. BI tools are dependent on these efforts.

Existing BI solutions are designed primarily for people who can understand the data models and who have time to build analyses from them, recall them for future use and provide information for others. Within most organizations, these experts account for about 5 percent of the salaried workforce. That was fine for a decade or so while we worked out the problems and fine-tuned the architectures and methodologies, but the people who are supposed to be served by this effort have largely voted with their feet and marched back to shadow systems, particularly spreadsheets, where the bulk of analytical work still takes place. While the number of BI users is increasing, the bulk of the increase is in passive report distribution, not active analysis, collaboration or decision-making.

Along with data, reporting drives BI. What is a report? At the simplest level, a report is the rendering of information requested from existing data, with at least some level of formatting and usually some added calculations, such as subtotals and totals at a minimum. Reports are read-only. Originally, reports were distributed on either paper or microfiche, but about 25 years ago it became possible to see them electronically. Today, they can be delivered to the Web, to portals, to cell phones, PDAs -- just about anywhere.

Making reports "interactive" doesn't really change their nature. The ability to select parameters, for instance, is actually a reporting application surrounding just another report. The ability to navigate through the elements of the data model interactively, such as OLAP, ultimately ends up as the visual display of just another report. No matter how many ways the fetching and formatting of data is embellished, it is still a report. Some OLAP tools are not read-only, but their use represents a tiny fraction of the BI use which, in turn, is an almost imperceptible fraction of the daily use of alternatives, especially spreadsheets.

BI 2.0: What to Expect

The new era of BI, which is already here, goes far beyond data and reporting. BI is becoming proactive, real-time, operational, integrated with business processes, and extending beyond the boundaries of the organization. To become pervasive and grow out of its reporting niche, BI has to provide simple, personal analytical tools on an as-needed basis with a minimal footprint and cost. Rather than relying solely on a rigid metaphor like data warehousing, BI needs the ability to access data anywhere it can be found and to performing integration on the fly, if necessary. Locating the right information to solve problems must be a semantic process, not requiring knowledge of data structures or canonical forms. Directed search, based on the meanings of and relationships among objects, allows practically any person or service to find what is needed without assistance, whether it is structured information like relational databases, message queues, logs, web services or even unstructured content.

The Web was not just a brilliant idea, it was and is a world-changing phenomenon. Yet even in light of this success, it is quickly moving to an entirely new plane of existence, one that is more collaborative, intelligent, rich, semantically aware and simple. Web 2.0 is about closing the loop, from being a collection of web pages to a being the single platform for just about everything. The physical resources are in place and attractively priced, the open standards are evolving without fracturing the confederacy of technology players, large and small. The missing piece is the semantic technology, but even that is progressing nicely as semantics pure plays are gobbled up by more well-heeled operations like IBM, Google and webMethods.

BI needs a 2.0 initiative too. BI 2.0 won't replace BI, it will extend it. Current BI capabilities are well-suited to many tasks, such as individualized dashboards of overnight data or providing a rich environment for interactive exploring of data, by people and by algorithms. But current BI practices have always been weak dealing with the unexpected and the urgent because of their "data first" orientation. Spreadsheets shine here, but they lack too many architectural prerequisites to be useful, such as collaboration, security, abstraction and versioning, among others. If spreadsheet tools develop these characteristics, and they are heading in that direction, they will not usurp BI, they will become BI.

Game Changers

BI 2.0 will disrupt the BI industry in quite a few ways, and foot-dragging can be expected. Some more obvious effects will be:

Close the Loop

Today, analytics is a singular process. An individual views a report in some form or creates a query to understand something. No matter how sophisticated or nave, the result is always the same (assuming it works); the analyst has informed his or her opinion or hypothesis, but the analytical tool stops there. It isn't possible to replay the steps and show others how the question was resolved. Nor is it possible, without a custom-made application, to transfer this new knowledge to a system or service that can act on it immediately. Nor is it possible -- again, without building an application -- to track the results of the decision explicitly (as in, "Orville, the action you took on pricing based on the new parameters that entered the system on Tuesday are showing a marked improvement in on-time arrivals at the terminal."). Delivering such capabilities would bring a Nirvana of decision support, something we have all been envisioning for a decade or more, but BI in its current state is not designed to close the loop. That's the promise of BI 2.0, and, in fact, it's the single driving reason for this new era to emerge.

Try New Thinking

Learning new skills is important and it's easy. What's difficult is wrapping your brain around all of this and getting ready to walk away from what you now accept as the correct approach. For instance, here are six BI 1.0 fallacies that fall by the wayside in the BI 2.0 era:

Prepare for the Inevitable

BI 2.0 is a natural evolution, but it isn't happening in a smooth, continuous fashion. It's more tectonic; forces build and changes happen abruptly. Enterprise technologies tend to be sticky because of their cost -- not just to implement but also to replace. However, all enterprise applications are following the lead of the Web, especially Web 2.0. With billions of daily users, the Web is the cohort for a massive longitudinal study. The results are in. People love collaboration, social networks, mash-up's, no-cost software, versionless software and the ability to get what they need without a lot of ceremony. BI 2.0 is inevitable.

What should you do to prepare? Mostly, readjust your thinking so you are open to the best offers. Here are six ideas that will give you to start:

BI is not a failure and it's not dead, either. BI 1.0 provided a great deal of utility, but the next wave will be hoisted up, as Issac Newton described his superhuman accomplishments, like "standing on the shoulders of giants." So climb on. The view is great.

Neil Raden is the founder of Hired Brains, providers of consulting, research and analysis in Business Intelligence, Performance Management, real-time analytics and information/semantic integration. Write him at [email protected].