A common mistake made during second-generation product selection is to evaluate against existing functionality that doesn't accurately reflect current (and certainly future) functional requirements. At one company, we encountered a BI environment largely centered on "The Management Dashboard" — a ubiquitous application that in reality functioned like a quasi-portal, with single-interface access to an assortment of reports and analyses covering almost every functional area in the enterprise. It had no true architecture; rather, it had evolved with the addition of components put in place to meet specific needs of the moment. Rather than evaluate replacement candidates against the current backdrop of wildly diverse functionality, we looked at new functional requirements that were closely aligned with the company's strategic business goals as the foundation for evaluating new ETL and user-facing tools.
Finally, when setting up the vendor Proof of Concept, include the current functionality that you want to retain. But make sure you highlight the differences among toolsets for essentially the same capabilities.
Other key factors, beyond technical issues, are important to consider. First, vendors fervently guard their "turf": They'll do everything they can to leverage existing relationships with key decision makers. You'll likely hear the ROI pitch, which emphasizes quantifiable benefits of staying the course with the current product investment — even if some glaring product shortcomings have tarnished the vendor's reputation among those who work most closely with its product.
At the same time, however, nonincumbent vendors covet "converted references": that is, organizations that have dumped rival products in favor of their own. If it makes sense to become a converted reference, don't forget that your conversion could give you some leverage in pricing negotiations.
Second, you'll see that pricing strategies vary considerably among BI software vendors. It's hard to count on even apples-to-oranges; you're more likely to face an "apples-to-end table" comparison. We recently issued a Request for Solution (RFS) document to five leading BI tool vendors for a second-generation implementation. From the exact same set of explicitly defined requirements for user population and functionality, pricing ranged from $750,000 to $15 million! Analyzing the vendors' responses, it was clear that the wide disparity was due to the challengers trying desperately to unseat two incumbent vendors included in the evaluation.
A final pricing consideration to look out for is the up-front concessions and discounts that come only at the cost of excessive, back-loaded maintenance costs or expenses for upgrades and enhancements. In the RFS situation just mentioned, one nonincumbent actually refused to provide three-year maintenance pricing in concert with its lowball estimate. The vendor wrote that it wouldn't "provide maintenance pricing at this time and [would] do so only during final pricing negotiations."
In the sidebar "Vendor Evaluation: Words From the Wise," we've summarized a few "best practices" points to remember as you embark on vendor evaluations for second-generation BI projects. Here, we'd like to close with a few final recommendations about tool evaluation.
- Set up some portion of your Proof of Concept demonstrations to emphasize what it will take to convert from the current environment. Note that this step doesn't apply only to challengers but also to incumbents: Because today's vendors are bidding with their next-generation products to replace legacy tools (for example, Cognos ReportNet to replace Impromptu), conversion is important regarding all vendors.
- Watch out for "renegade applications" running with the existing toolset that pose a hidden conversion threat because of complex interfaces and other factors.
- Minimize bias and promote objectivity and fairness by addressing history and politics. However, don't let these factors become the overarching criteria.
As BI environments mature, most organizations will find fewer and fewer areas where they can start with a clean slate. The good news is that more modern technology holds the opportunity of getting you closer to achieving your objectives. The trick is to move forward with open eyes, wiser from experience, and ready to balance all the factors to achieve a successful conclusion.
Steve Robinson is the Business Intelligence practice director for Verity Partners, LLC and co-author of Principles and Practice of Information Security (Pearson Education, 2003).
Alan Simon is vice president of the Business Intelligence Practice at Alliance Consulting; he is the author of Data Warehousing for Dummies (Wiley, 1997) and was the data warehousing columnist for Database Programming & Design.
VENDOR EVALUATION: WORDS FROM THE WISE