Dashboarding Ourselves - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management
11:27 AM

Dashboarding Ourselves

Enough self-serving reports: It's time for honest self-assessment about why projects are falling short of their objectives.

The data warehousing/business intelligence (DW/BI) industry is largely self-evaluating — those with the most to gain from favorable reviews are usually also in control of reporting results. Vendors naturally desire to emphasize the most positive and dramatic outcomes, organizations that spend large sums of money are reluctant to publicize failing or underperforming implementations, and analysts walk a fine line between informing their corporate customers and not provoking their vendor patrons. There are some fiercely independent sources of information, such as Nigel Pendse's OLAP Report, but sadly, it only covers a fragment of the market.

In the recent past, some analysts issued reports claiming that extract, transform, and load (ETL) was far superior to alternatives, but ETL vendors sponsored the study. Another study evaluated ROI and TCO for various ETL vendors and, not surprisingly, the ones that demonstrated the best results were also clients of the analyst firm. Many studies employ a shaky methodology, such as the famous Data Warehouse ROI report by IDC (1996 Foundations of Wisdom study), that found an average ROI of 401 percent, but a closer examination revealed that those with ROIs less than zero were excluded from the calculation. A more recent study by TDWI, one of the leading data warehouse education and research organizations, disclosed that almost 70 percent of its survey participants were IT, vendors, or consultants, not stakeholders in organizations who ultimately benefit from these initiatives.

As a DW/BI practitioner, I found these findings very suspicious as they didn't track with my experiences at Hired Brains: They were simply too optimistic and, in some cases, seemed contrived. Despite our best efforts, projects all too often fail to reach their goals for reasons that may not be obvious. It's easy to pinpoint the usual suspects, such as mid-project organization realignment and killer politics, our inability as consultants to convince our clients that certain decisions are suboptimal and a host of others, well documented in the literature (such as TDWI's "Ten Mistakes to Avoid" series). But there are also many cases where everything goes well, yet the initiative never gets traction in the organization, penetration stays at a very low level, and ROI projections aren't met. In many of those cases, this failure to thrive is something of mystery.

To understand how to help our clients achieve success with this technology, we at Hired Brains decided to go to the source and survey the actual BI stakeholders and try to understand the phenomenology of getting BI to work. What we found was very revealing See Figure 1 here.

Basically, as an industry, we've mastered the technology, but we've failed at a cognitive level. We haven't been able to change the way people use information, particularly their reliance on standalone spreadsheets and personal databases. (See the sidebar, "In the Shadows.") This single finding not only explains the low penetration rate of BI but also, more ominously, torpedoes any hope of a good ROI, as people only have so much time in a day. If the time spent with desktop tools doesn't decrease, then the benefits from the BI effort won't be realized. We also failed to provide tools that are indispensable or even very useful. The relevance, integration, understanding, and workflow elements of our offerings are all sorely lacking. How can this be?

The Training Mindset

There are a few clues in the survey data, although they're only that; more in-depth research is needed and ongoing. Training clearly has a direct relationship to the perception of success. As the chart shows (see "Training Spend Vs. Outcome" in Figure 1), organizations that devote at least 16 percent of the project budget on training have a greater than 90 percent favorable rating for the data warehouse effort. Organizations that spend 5 percent or less, a pretty typical investment, have a largely unfavorable rating.

Clearly, just spending money on training is insufficient for achieving success, but those organizations that invest in training expose a mindset — cooperation between technology and people and taking an interest in the work that people do — that affects all their decision-making. In those cases where training is given only a few dollars as an afterthought, stakeholders largely find the whole initiative irrelevant. It isn't the training itself that counts, it's what it denotes: An organization that takes an interest in the work that people do and forms a partnership to continuously improve the tools and the output by weaving technology into the work environment, not imposing it.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 3
Comment  | 
Print  | 
More Insights
COVID-19: Using Data to Map Infections, Hospital Beds, and More
Jessica Davis, Senior Editor, Enterprise Apps,  3/25/2020
Enterprise Guide to Robotic Process Automation
Cathleen Gagne, Managing Editor, InformationWeek,  3/23/2020
How Startup Innovation Can Help Enterprises Face COVID-19
Joao-Pierre S. Ruth, Senior Writer,  3/24/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
IT Careers: Tech Drives Constant Change
Advances in information technology and management concepts mean that IT professionals must update their skill sets, even their career goals on an almost yearly basis. In this IT Trend Report, experts share advice on how IT pros can keep up with this every-changing job market. Read it today!
Flash Poll