Roelant Vos

Data solution design patterns, implementation, and automation

Category: Architecture

This category contains all documents regarding BI, DWH, ETL and Front-end architecture. Also includes datamodelling (Data Vault, Inmon, Kimball).

0

Architecture

Guidelines for Data Integration Processes

The following are core principles I use for implementing data integration / logistics (‘ETL’). This is a component (design pattern) from the Data Solution Framework repository, specifically this pattern. The purpose of this pattern is to define a set of minimal requirements that every single data logistics process (i.e. procedure, mapping, module, package, data pipeline) should conform to. These fundamental guidelines direct how all data integration processes should behave under the architecture of the Data...

0

Architecture / ETL

Major revision of the DIRECT framework

A new set of improvements have been committed to the data logistics control (‘ETL process control’) framework Github. This framework, referred to DIRECT (thanks to acronimify.com), assumes the spot of the Data Logistics Process Control in the engine metaphor for flexible data platform management. In case you were wondering, it’s the Data Integration Run-time Execution Control Tool! This is one of my favourite components, as it’s both so simple but also hard to truly get...