The Importance of Agile Business Intelligence

Don't be misled by the title. This isn't about Agile business intelligence -- it's about agile business intelligence, which is not necessarily the same thing.

Just in case you don't see the difference, Agile business intelligence is doing BI the way of the Agile community -- as stubbed in this Wikipedia entry. In a nutshell, it is applying the popular Agile methodology (or a version of it) to business intelligence.But whether one is a believer/adherent of the Agile methodology or not, there is no escaping the need to be "agile" on any BI project -- as defined by, say, Merriam Webster (marked by ready ability to move with quick easy grace; having a quick resourceful and adaptable character) or Microsoft Encarta (nimble; able to move quickly and with suppleness, skill, and control). I like the Encarta definition better -- note the inclusion of "control."

The main reason a BI project needs to stay agile is that business requirements are harder to specify for BI than for, say, a conventional Web application. It's true -- although trite -- that all business requirements change/evolve (to a varying extent) over time, regardless of the type of software we're building. Trouble is, BI requirements are harder to pin down in the first place.

That's because by its very nature, business intelligence is really about providing a software extension to the user's (analytical) thought processes. If I'm analyzing a vendor, what exactly does that mean? What information do I need to analyze the vendor? What sort of visual presentation would best serve my purpose? What related information would be helpful?

These questions will likely get different answers from different users, because people don't think alike. So what are we to do? Limiting ourselves to a common denominator approach would likely be so bland as to not produce much value. On the other hand, a particularly fanciful user's needs (e.g. getting this one extra data field from an application that we are not otherwise integrating) may shoot up the cost and prove hard to justify.

A sound approach is both-sides-to-the-middle: select data sources and explicitly eliminate the rest -- this is the top-down part -- and grow outwards beginning with a common denominator attribute set -- this is the bottom-up part. This "growing outwards" is where the need for controlled agility comes in. Note: so far, this is all about ETL, which is about 60-80% of the complexity, cost and especially risk of the typical BI project. Broadly speaking, you could apply the same approach to the presentation as well.

How about using Agile methodology (with a capital A)? Sure, that's a good way to begin -- as long as you are able to customize it to the needs of the project and don't get too dogmatic about it. If you haven't yet realized that there are no silver bullets in IT, you probably haven't been at IT long enough.

Or perhaps I'm a little too jaded myself...Don't be misled by the title. This isn't about Agile business intelligence -- it's about agile business intelligence, which is not necessarily the same thing. Just in case you don't see the difference...