Through our education and consulting practices, the Kimball Group has been exposed to hundreds of successful data warehouses. Careful study of these successes has revealed a set of extract, transformation, and load (ETL) best practices. We first described these best practices in an Intelligent Enterprise column three years ago. Since then we have continued to […]
Conformed dimensions are the glue that ties together your enterprise data warehouse. To facilitate and manage the conforming process, we have identified two additional fundamental responsibilities for the DW/BI team: the dimension manager and fact provider. Typically these functions are performed by the ETL team working closely with the data stewardship organization. The dimension manager is a centralized […]
Best practices are precision tools that should be wielded precisely and skillfully. This article describes five best practices drawn from the Kimball Method that often are described incorrectly. Data warehousing is a mature discipline with well-established best practices. But these best practices are useless or even harmful if they are described inaccurately or incompletely. One […]
Meaningless integer keys, otherwise known as surrogate keys, are commonly used as primary keys for dimension tables in data warehouse designs. Our students frequently ask us – what about fact tables? Should a unique surrogate key be assigned for every row in a fact table? Although for the logical design of a fact table, the answer is no, […]
Consistent data is the Holy Grail for most data warehouse initiatives, and data stewards are the crusaders who fearlessly strive toward that goal. An active data stewardship program identifies, defines and protects data across the organization. Stewardship ensures the initial effort to populate the data warehouse is done correctly, while significantly reducing the amount of […]
Your ETL system may need to process late arriving dimension data for a variety of reasons. This design tip discusses the scenario where the entire dimension row routinely arrives late, perhaps well after impacted fact rows have been loaded. For example, a new employee may be eligible for healthcare insurance coverage beginning with their first day on the […]
In Design Tip # 69, Identifying Business Processes, Margy discussed the importance of recognizing your organization’s business processes and provided guidelines to spot them. We dive into more details here. Focusing on business processes is absolutely critical to successfully implement a DW/BI solution using the Kimball Method. Business processes are the fundamental building block of a dimensional data warehouse. […]
Many data warehouse teams lean heavily toward the doing side. They leap into implementation activities without spending enough time and energy to develop their data models, identify thorough business rules, or plan their data staging processes. As a result, they charge full speed ahead and end up re-working their processes, delivering bad or incomplete data, and generally causing themselves […]
It’s not unusual to identify dozens of different dates, each with business significance that must be included in a dimensional design. For example, in a financial services organization you might be dealing with deposit date, withdrawal date, funding date, check written date, check processed date, account opened date, card issued date, product introduction date, promotion begin date, customer birth […]
Business acceptance is a bigger problem than BI/DW professionals want to admit. Here’s how to get on the right track. A recent Data Warehouse Designer column, “Data Warehouse Check-Ups” (June 12, 2004), discussed the importance of regularly casting a critical eye over your data warehouse/business intelligence (DW/BI) program. Check-ups identify early warning signs and symptoms so appropriate […]