One of those hard problems is impact analysis and regression testing. When someone makes a change any where in the BI lifecycle, say a physical field in the data warehouse, identifying affected reports is often guesswork. A handful of BI vendors do "okay" in telling you which reports are impacted (see BIScorecard, architecture criteria for detailed scores); for most others, the reports will out and out break.At MicroStrategy's Symposium in New York this week, the vendor previewed Integrity Manager, due for release in September. Integrity Manager goes beyond simple impact analysis, providing full regression testing for any number of potential system changes (operating system change, VLDB settings, metric calculations, data warehouse load errors, and so on). Integrity Manager will do before-and-after report comparisons to tell an administrator the impact of the change, warning if there are differences in the data values, if the SQL generated is different, if the output format is affected. Not only will Integrity Manager tell you that there is a difference, it also highlights the differences in the numeric values, SQL, or display.
I have not seen anything near this capability in other leading BI suites. It's one of those details that few business users will get excited about, and yet one that can be a hot button for BI administrators. Integrity Manager can help these BI administrators know in advance which users and reports will be affected, and it can forewarn them when a change outside their area of control impacts BI.
As BI increasingly becomes an enterprise (vs. departmental) resource, such integrity checking and proactive alerting is important. Today, most BI administrators are left with either no testing or tedious, manual tests. MicroStrategy is definitely ahead of the market in this capability, showing that what it may not have in terms of breadth, it more than makes up for in BI depth.
Cindi Howson Founder, BIScorecard product evaluationsWhile much of the BI market has been busy expanding its solution breadth - acquiring or developing performance management capabilities - one vendor that has stayed focus exclusively on the BI front-end has been MicroStrategy... The company maintains that there is still a lot of work to be done to solve hard problems in the traditional BI space... problems such as impact analysis and regression testing.