The ‘architecture’ section of this weblog serves as an introduction for the Data Integration Framework Github repository (see the collaboration section). This Github repository contains the various levels of documentation that constitutes the Enterprise Data Warehouse (EDW) architecture, pattern documentation and supporting metadata and control frameworks.
Design and Solution Patterns for the Enterprise Data Warehouse
Patterns are design decisions, or patterns, that describe the ‘how-to’ of the Enterprise Data Warehouse (and Business Intelligence) architecture. They specify the rules the architecture has to play by, and they set the stage for (future) solution development. Design and Solution Patterns play a large role in the data integration and Data Warehousing ecosystem (have a look at the collaboration section what is underway). The following diagram explains their role in the context of delivering an Enterprise Data Warehouse solution:
Every layer in the architecture specifies which concepts and steps are to be carried out in that particular layer, but how to exactly define this is specified in the Design Patterns. The Design Patterns are therefore both the starting point for the solution design as the main tool of the Data Warehouse architect to maintain the system.
Design Patterns are fundamental concepts and contain (and explain) the design decisions and considerations made. They therefore have a huge impact on the technical implementation of the Data Warehouse.
A Design Pattern is usually not tool specific but can it can be; depending on the nature of the specific concept explained. It is advised however to keep the Design Patterns as detailed as possible without including tool dependencies. This way they will remain true even when the specific software solution (implementation) changes.
Examples of Design Patterns are the listing of essential ETL requirements, handling logical deletes, key distribution and managing temporal / time-variant information. Design Patterns can also apply to specific methodologies, for instance loading Satellite data in a Data Vault context.
Design Patterns are managed in the Data Integration Framework Github. Send me an email if you are interested in collaboration.
Solution Patterns are highly detailed implementation documentation for specific software platforms. Typically a single Design Patterns is referred to by multiple Solution Patterns, all of which document how to exactly implement the concept using a specific technology.
Similar to the Design Patterns, the Solution Patterns are managed in the Data Integration Framework Github. Send me an email if it makes sense for you to have a look.