Comments from Business Intelligence Pipeline readers illuminate how critical an issue data quality has become in the world of BI.
We asked you for feedback on the data quality issue, and Business Intelligence Pipeline readers responded in a way that illuminates how critical the topic has become in the realm of BI. Readers not only shared their frustration with managing data quality, but suggested a few fixes.
Budgetary limitations and long work backlogs were the two factors most frequently cited by readers as contributing to data quality problems. In short, IT has neither the funding nor the time to address data quality as much as it wants to. So what are organizations doing to overcome the challenge? We got a lot of solid feedback, much of it relatively simple, but effective.
An IT manager in state government said his department raises the priority attached to bad data by reporting relevant data problems to internal auditors and IT security managers. When those people get involved, data problems are more likely to get addressed. The same reader said giving business users greater access to data is often instrumental in uncovering and highlighting data problems. "When they encounter the deficiencies and realize the impact, they are much more likely to request or actively participate in the data cleanup," he said.
Improved training of data input operators came up with several readers. A reader with 10 years of experience as a consultant said untrained, unskilled workers are increasingly handling data entry due to cost constraints, which is worsening the data quality problem.
Readers' comments clearly indicate that meta data quality is highly important, though one reader said the concept of good meta data at his organization isn't always widely accepted from department to department.
Educating business users and management about the importance of data quality came up more than once, though few readers shared specifics on how to go about explaining the situation to those people. It's my hunch -- and let me know if you disagree with me here -- that a lack of communication about data quality between IT and business users is the single biggest stumbling block to fixing processes that lead to bad data.
"I don't think IT can divorce itself from data quality issues," said one IT professional. "I do believe that users of data need to participate in the improvement. Improved data quality [that comes from] the elimination of redundant data and the steps that cause errors is important to IT and user departments."
Thanks to everyone who wrote in with their thoughts on this topic. I welcome any other input readers have to share -- including responses to the comments I posted here. You can e-mail me here.
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.