"In all affairs it's a healthy thing now and then to hang a question mark on the things you have long taken for granted. " --Bertrand Russell
In the next five years, business intelligence as we know it will disappear.
To be more precise, BI as we know it will disappear from view. Existing BI will linger, but the real action will be elsewhere. In an accelerating arc of progress with enterprise systems, organizations are moving beyond purely operational systems and toward embedding analytics into operational processes. The need to take action on analytics, not just be informed by them, has never been more urgent. The relentless externalization of business, the rapid emergence of loosely coupled computing environments based on standards and the Web-as-the-platform paradigm are all driving BI in a new and exciting direction. This is an extraordinary opportunity for the BI community, but only for those who can adapt to the changing circumstances.
We'll take a fresh look at BI "megatrends"--the major drivers that are impacting how you'll architect and implement BI in your organization. Guided search, master data management, Web services, semantics and open source are just some of the exciting topics that we'll discuss in this year's tour of the changing BI landscape.
Last year's megatrends article predicted that BI would be dominated by a few large players. One year later, some of those players are struggling and many new ones are emerging to address the gaps in conventional BI. Open-source vendors like Pentaho and JReport are making their presences felt. Actuate provides BIRT (Business Intelligence Reporting and Tools), a report-authoring tool, based on the open-source development environment Eclipse. Endeca Technologies, which offers a guided search, navigation and analysis platform, threatens to redefine the meaning of BI by concentrating on the user experience instead of the data.
Celequest is breaking new ground by delivering real-time agent technology with analytical capability, and recently, offering an all-in-one BI appliance. And through interactive visualization and data manipulation, Spotfire is vastly expanding what users can do with analytics. Fueling the momentum toward software as a service (SaaS), Sharp Analytics and similar companies are providing expert-level domain knowledge bundled with software and services for a flat fee.
Small companies aren't the only innovators. SAP's NetWeaver architecture encourages third parties to add value to the SAP environment by developing hybrid, operational/analytical applications as combinations of services. Database vendors are incorporating analytics and also developing strategies or even new products for SOA. Farthest along is NCR's Teradata Division with its Teradata Application Protocol (TAP); it lets Web Services leverage Teradata data-warehouse engines.
Microsoft is moving its desktop tools, particularly Excel, into a Web Services format. Office 2007 will bring Excel Web Services, Excel integration with SharePoint portal and HTML output for viewing spreadsheets and manipulating them through a browser. Microsoft is tightening BI and Reporting Services integration with its SQL Server and Analysis Services. As its PerformancePoint plans unfold, the company will incorporate the proven capabilities found in the acquired ProClarity tools.
Typically, BI has been a disconnected activity, with users unable to traverse the big gap between being informed and being able to do anything about it. Currently, stacks of BI functionality, such as reporting, ad hoc analysis and OLAP (online analytical processing), serve only a small fraction of the user community (as measured by licenses)--10 percent to 20 percent, according to most studies, though actual use is probably even lower. Although the tools generally require no programming and the basics can be taught in a couple of days, conceptually BI still demands a level of understanding about the data, data models and data manipulation processes beyond most knowledge worker's skills or patience. Consequently, the level of IT involvement in BI is still high, leading to reduced ROI and an overall lack of agility and timeliness, except for those few power users who invest the time and effort to develop proficiency.
Despite these drawbacks, there are many BI success stories. It would be unfair to call BI an all-out failure. What factors become evident when you look at BI successes? First is an unusually receptive company culture, often driven to BI by pressing crises that only BI technology can address. The second factor is where everything is simply done right: that is, excellent application of the right practices along with management encouragement. Unfortunately, such conditions are not prevalent in most situations, causing BI to suffer from a "failure to thrive."
The BI community is replete with methodologies, layered architecture diagrams, technical product reviews, secrets of success, and mistakes to avoid. However, most of what's discussed focuses on the "plumbing" enablers: data movement, quality, integration, management and modeling. This stuff more properly falls under the heading of "data warehousing." What we should be talking more about is simply what real people need from BI to do their work.
Now, Adam Smith's invisible hand is moving; the BI market is beginning to shift as the players bring new opportunities to thrive into focus. Some of the largest vendors perceive this movement and are reacting. Smaller players are pressing their advantages and making their presence known.