In most cases, metadata is a neglected area of the DW/BI system; however, an increasing number of DW/BI teams have made positive strides in delivering business metadata to their users. This Design Tip looks beyond the business metadata to suggest several opportunities for leveraging ETL process metadata to improve data warehouse operations. The goal is […]
ETL and Data Quality
Whether you are developing a new dimensional data warehouse or replacing an existing environment, the ETL (extract, transform, load) implementation effort is inevitably on the critical path. Difficult data sources, unclear requirements, data quality problems, changing scope, and other unforeseen problems often conspire to put the squeeze on the ETL development team. It simply may […]
This article describes six key decisions that must be made while crafting the ETL architecture for a dimensional data warehouse. These decisions have significant impacts on the upfront and ongoing cost and complexity of the ETL solution and, ultimately, on the success of the overall BI/DW solution. Read on for Kimball Group’s advice on making […]
Through our education and consulting practices, the Kimball Group has been exposed to hundreds of successful data warehouses. Careful study of these successes has revealed a set of extract, transformation, and load (ETL) best practices. We first described these best practices in an Intelligent Enterprise column three years ago. Since then we have continued to […]
Conformed dimensions are the glue that ties together your enterprise data warehouse. To facilitate and manage the conforming process, we have identified two additional fundamental responsibilities for the DW/BI team: the dimension manager and fact provider. Typically these functions are performed by the ETL team working closely with the data stewardship organization. The dimension manager is a centralized […]
Your ETL system may need to process late arriving dimension data for a variety of reasons. This design tip discusses the scenario where the entire dimension row routinely arrives late, perhaps well after impacted fact rows have been loaded. For example, a new employee may be eligible for healthcare insurance coverage beginning with their first day on the […]