Transforming An Antiquated Business Intelligence Process

12 Ways To Connect Data Analytics To Business Outcomes

Five years ago, I was leading IT strategy and innovation for a Fortune 500 global chemical company. One of my tasks was to improve our business intelligence, but I had a tough problem. I noticed key decision makers weren't viewing BI reports. Managers used the reports to monitor past performance and track their results against key performance indicators, but executives were not using them at all. I needed to rethink my reports and give executives something they could use.

When I asked the CFO how we could improve our reports, I realized we were doing things all wrong. The charts and graphs our business intelligence tool produced contained internal historical data – events our decision makers and finance team could not change. Instead, our CFO needed forward-looking demand drivers specific to our company – predictive insights that our finance team could actually act on. That was the moment I realized the most important information was where our business was going, not where it had been.

The true business drivers for the company required a combination of internal metrics and external micro- and macro-economic factors. Without combining these metrics to create a clearer picture of performance, the C-suite deemed our reports useless.

An antiquated process

I soon realized that this problem wasn't specific to our company. But, like most companies, a new idea is approached very cautiously and often is met by internal resistance. I knew I first had to build and test my theory to prove this kind of approach would indeed improve our ability to forecast.

I decided that this was important enough to do on my own time to avoid being trumped by other internal projects and day-to-day tasks. Once I could demonstrate the validity of this approach, I knew it was only a matter of finding the right person inside the organization who would champion this integration. So I began researching the issue, meeting with the chief economist of a chemical association who confirmed that we were indeed missing the leading external signals of chemical demand.

At the time, corporations that wanted to analyze the impact of external factors on their performance had to pay a heavy price for a skilled economist to use statistical modeling tools that were 20 to 30 years old and took months of data hunting and gathering followed by tedious analysis. The process was largely based on trial and error – guessing which external factors might impact your business, and then running manual tests to determine if that guess was correct.

Plus, because external data is constantly changing, these models were out of date as soon as they were developed. During a severe economic crisis, global companies were struggling to understand the fast-moving economy, and without the right set of tools, the problem was only getting worse.

Real-time data correlation

Over the last 10 years, extremely valuable global data on economic conditions, consumer behavior, and other useful data sets have become more readily available. The difficult process was building an infrastructure that could sift through millions of data sets to identify leading factors related to a company’s internal performance. This challenge is why more companies aren't harnessing the power of external data. The metrics are available in mass quantities, but how do businesses make sense of them all?

I constructed an in-memory correlation, pattern-matching, and forecasting engine that gathered third-party data from millions of data sets. It calculated if and how each external factor specifically impacted each internal performance metric, and to what exact degree. The end result was fact-based models of leading indicators that explained how much loss or gain a business would experience depending on its specific external performance drivers – such as the average amount of rain, or an increase in real estate prices in particular regions.

This solution improved our company's forecast accuracy for our largest division and provided an automated, systematic, and repeatable platform for a process that had always been manual, tedious, and often overlooked. Once we implemented this process, our executive team was able to perform market analysis in just a few hours; a process that used to take weeks and months to configure. Ultimately, we were able to confirm or deny several long-held assumptions about our business performance. For instance, we were able to quantify one specific external factor to determine exactly how long it would take to impact our business, and by how much. In other cases, we learned that some factors didn't affect our business as much as we believed, and were able to concentrate on those that truly mattered. These insights were integral in preparing the board and investor meetings.

[ And here is another transformation story. Read NFL CIO: Football's Digital Transformation. ]

Asking Why

As is the case with most large companies, not every department adopted this new solution over night. Yet, for our finance team and executives, the insight gleaned from this predictive analytics process quickly became a key part of the planning process. These changes wouldn't have happened if I hadn't asked myself, Why? Why weren't our executives using our reports? Why did we only operate using internal data? Why have we been using the process for years?

Questioning the validity of a widely used business intelligence process was not easy. It required that I spend a lot of my personal time looking for answers, but questioning processes that have been in place for years is where true innovation starts. While an entrepreneur may have to work long nights and go through rounds of failed attempts, in the end, the most rewarding part of innovation is not only seeing your hard work pay off, but also knowing that you've solved a major pain point for global businesses. Today, businesses can proactively address challenges and opportunities, ultimately improving bottom line profits. It all started with asking, Why? which transformed an antiquated process from the inside out.

Does your company offer the most rewarding place to work in IT? Do you know of an organization that stands out from the pack when it comes to how IT workers are treated? Make your voice heard. Submit your entry now for InformationWeek's People's Choice Award. Full details and a submission form can be found here.