Smarter The Second Time

Technical Requirements

No matter how thoroughly your organization analyzed data volumes, response time, and other technical requirements when the original tools were selected, it's essential to completely reassess those requirements to establish a more current baseline. In the world of data warehousing, some reasons are obvious: You might have new data sources and an expanding user community. More subtle reasons could include the following:

Security needs are another critical factor. Very often, first-generation BI environments were built and deployed with minimally acceptable security standards set by what any mainstream toolset could provide. After all, part of the purpose was to make data more accessible. Now, with organizations typically much smarter and more demanding about security, second-generation BI systems must be better. BI products vary considerably with respect to security models. Choose your new toolset based on today's and tomorrow's security needs, not yesterday's.

Additionally, since you deployed your current BI system, the core technologies that comprise commercial software products — BI or otherwise — have almost certainly evolved. Very likely, your organization has some sort of "architectural blueprint" that governs permissible standards and platform technologies for all new applications; the blueprint offers a "playbook" of allowable interfaces among systems and components. If, for example, Web services, a common portal, and directory services integration are now required capabilities for all new deployments — whether transactional or analytic — then the technical compatibility requirements factored into your BI tool evaluation need to reflect this changing landscape.

Since BI is an end-to-end proposition, it's likely that both the ETL and user-facing tools (reporting, OLAP, and so forth) must conform as a group to a set of forward-looking technical standards. The key point here is that you proceed at great peril if you undertake second-generation tool evaluation and selection based on an obsolete set of technical requirements.

New Functional Requirements

More often than not, first-generation BI applications do little beyond producing a mixture of standard and parameter-driven, after-the-fact reports and analyses that can't be altered without IT involvement. This lack of flexibility won't satisfy today's decision makers who value self-service and want the ability to look ahead. Most likely, your organization now employs several packaged applications. The desired data must pass through a new generation of ETL tools and reach users via the new BI tools that you're evaluating. While it's clear that you'll need to factor in issues related to these new classes of data, don't neglect the metadata issues associated with the new data sources in your evaluation.

A critical shortcoming of most first-generation BI environments is inadequate metadata management, due in large part to the product-centric nature of most early tools. Develop a broader and bolder vision of the role metadata management will play in your future BI environment; thoroughly evaluate the candidate tools to see if they can implement that vision.

Many organizations now have functional requirements that call for real-time data flows from source systems into the BI environment. This becomes more than just a technical issue; the business success of the BI environment may depend on recognizing that, for example, the widely used "Daily Sales Activity Report" currently produced overnight must now be available, on demand, several times a day. Therefore, real-time data must be included for users across the enterprise. This sort of data demand has far-reaching implications not only for extract, transform, and load (tools must support multiple interface protocols between source systems and the data warehouse), but also for reporting and analysis tools. Consider how the system will alert users as to the exact "age" or latency of the information they're analyzing. You'll also need to think about the BI system will update information in OLAP cubes and caches.

A common mistake made during second-generation product selection is to evaluate against existing functionality that doesn't accurately reflect current (and certainly future) functional requirements. At one company, we encountered a BI environment largely centered on "The Management Dashboard" — a ubiquitous application that in reality functioned like a quasi-portal, with single-interface access to an assortment of reports and analyses covering almost every functional area in the enterprise. It had no true architecture; rather, it had evolved with the addition of components put in place to meet specific needs of the moment. Rather than evaluate replacement candidates against the current backdrop of wildly diverse functionality, we looked at new functional requirements that were closely aligned with the company's strategic business goals as the foundation for evaluating new ETL and user-facing tools.

Finally, when setting up the vendor Proof of Concept, include the current functionality that you want to retain. But make sure you highlight the differences among toolsets for essentially the same capabilities.

Vendor Strategies

Other key factors, beyond technical issues, are important to consider. First, vendors fervently guard their "turf": They'll do everything they can to leverage existing relationships with key decision makers. You'll likely hear the ROI pitch, which emphasizes quantifiable benefits of staying the course with the current product investment — even if some glaring product shortcomings have tarnished the vendor's reputation among those who work most closely with its product.

At the same time, however, nonincumbent vendors covet "converted references": that is, organizations that have dumped rival products in favor of their own. If it makes sense to become a converted reference, don't forget that your conversion could give you some leverage in pricing negotiations.

Second, you'll see that pricing strategies vary considerably among BI software vendors. It's hard to count on even apples-to-oranges; you're more likely to face an "apples-to-end table" comparison. We recently issued a Request for Solution (RFS) document to five leading BI tool vendors for a second-generation implementation. From the exact same set of explicitly defined requirements for user population and functionality, pricing ranged from $750,000 to $15 million! Analyzing the vendors' responses, it was clear that the wide disparity was due to the challengers trying desperately to unseat two incumbent vendors included in the evaluation.

A final pricing consideration to look out for is the up-front concessions and discounts that come only at the cost of excessive, back-loaded maintenance costs or expenses for upgrades and enhancements. In the RFS situation just mentioned, one nonincumbent actually refused to provide three-year maintenance pricing in concert with its lowball estimate. The vendor wrote that it wouldn't "provide maintenance pricing at this time and [would] do so only during final pricing negotiations."

Closing Advice

In the sidebar "Vendor Evaluation: Words From the Wise," we've summarized a few "best practices" points to remember as you embark on vendor evaluations for second-generation BI projects. Here, we'd like to close with a few final recommendations about tool evaluation.

As BI environments mature, most organizations will find fewer and fewer areas where they can start with a clean slate. The good news is that more modern technology holds the opportunity of getting you closer to achieving your objectives. The trick is to move forward with open eyes, wiser from experience, and ready to balance all the factors to achieve a successful conclusion.

Steve Robinson is the Business Intelligence practice director for Verity Partners, LLC and co-author of Principles and Practice of Information Security (Pearson Education, 2003).

Alan Simon is vice president of the Business Intelligence Practice at Alliance Consulting; he is the author of Data Warehousing for Dummies (Wiley, 1997) and was the data warehousing columnist for Database Programming & Design.