Data Quality: The Problem That Won't Go Away

Data quality is a problem for BI practitioners across the board, at organizations large and small. So what's the fix?

Only a fifth of respondents to our latest Business Intelligence Pipeline reader poll rated the quality of their enterprise data as "nearly flawless." Nearly half rated their data quality as "fair," saying their organizations need greater focus on getting good data. Another fifth said their enterprise data is "often useless."

Most IT managers involved in business intelligence have to overcome either inconsistent or worthless data, whether their BI efforts are targeted or enterprise-wide. Failure can result in analysis of limited value, or no value at all.

An IT manager from the hospitality industry wrote to me recently on the topic. "I feel that data quality is of the utmost importance due to the increasing reliance in enterprise supply chain software on accurate inventory and other statistical values for use in supply chain automation," he said. But like many IT managers with limited staff and finite resources, he finds fixing the problem a major challenge. "In my particular situation, data quality control is often put on the back burner in favor of other tasks that upper management views as a higher priority," he said. "This is, in my view, a serious problem."

Other Business Intelligence Pipeline readers have provided similar insights on data quality. I'd like to hear from more of you.

Where should IT managers place their emphasis to ensure that their enterprises have clean, consistent data that lends itself to solid analysis? How important is meta data? How much does the problem involve inadequate training of non-IT personnel? Is the problem getting better, or worse? Do you have any tricks of the trade that you'd like to share with other readers?

Drop me a line and let me know what you think. Include "data quality" in the subject line. I assure the anonymity of anyone who provides their insights. Data quality is a perpetually critical -- and often overlooked -- challenge for BI practitioners everywhere. A few best practices can help everyone overcome an issue that has become of central importance to successful business intelligence.