Kimball University: Eight Guidelines for Low-Risk Enterprise Data Warehousing
New data sources and BI delivery modes make it that much harder for EDW initiatives to succeed. Here are eight recommendations for controlling project costs and reducing risks.
In this era of financial uncertainty, it's hard to justify a classic "waterfall" approach to EDW/BI development. In the waterfall approach, a written functional specification is created that completely specifies the sources, the final deliverables and the detailed implementation. The rest of the project implements this specification, often with a big-bang comprehensive release. The origins of the waterfall approach lie in the manufacturing industry, where changes after implementation are prohibitively costly. The problem with the waterfall approach for EDW/BI projects is that it takes too long and does not recognize the need to adapt to new requirements or changes in understanding.
Many EDW/BI projects are gravitating to what could be called an "agile" approach that emphasizes frequent releases and mid-course corrections. Interestingly, a fundamental tenet of the agile approach is ownership by the business users, not by technical developers.
An agile approach requires tolerating some code rewriting and not depending on fixed-price contracts. The agile approach can successfully be adapted to enterprisewide projects such as master data management and enterprise integration. In these cases, the first few agile releases are not working code but rather architectural guidelines.
Start with Lightweight, Focused Governance
Governance is recognizing the value of your data assets and managing those assets responsibly. Governance is not something that is tacked onto the end of an EDW/BI project. Governance is part of a larger culture that recognizes the value of your data assets and is supported and driven by senior executives. At the level of an individual project, governance is identifying, cataloging, valuing, assigning responsibility, securing, protecting, complying, controlling, improving, establishing consistent practices, integrating across subject areas, planning for growth, planning to harvest value, and generally nurturing. Governance doesn't need a waterfall approach, but these issues need to be part of the project from the very start. Failing to think about governance can result in fundamental rework of the EDW/BI project.
Build a Simple, Universal Platform
One thing is certain in the BI space: the nature of the end-user-facing BI tools cannot be predicted. In the future, what's going to be more important: data mining predictive analytics, delivery to mobile devices, batch reporting, real-time alerts, or something we haven't thought of yet? Fortunately, we have a good answer to this question; we must recognize that the enterprise data warehouse is the single platform for all forms of business intelligence. This viewpoint makes us realize that the EDW's interface to all forms of BI must be agnostic, simple and universal.
Dimensional modeling meets these goals as the interface to all forms of BI. Dimensional schemas contain all possible data relationships, but at the same time can be processed efficiently with simple SQL emitted by any BI tool.
The Agile ArchiveWhen it comes to managing data, donít look at backup and archiving systems as burdens and cost centers. A well-designed archive can enhance data protection and restores, ease search and e-discovery efforts, and save money by intelligently moving data from expensive primary storage systems.
2014 Analytics, BI, and Information Management SurveyITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.