Major update for TEAM and Virtual Data Warehouse software
TEAM and VDW code generation software for Data Warehouse automation 1.6 release now fully aligned to generic schema for Data Warehouse automation!
Tagged: Data Vault
Easy generation of Data Vault Satellites using standard SQL
This new entry in the ‘simple ETL generation series‘ leaves the Data Vault Hub concepts behind. Much of the logic shared in earlier Hub ETL generation posts will be reused for the purpose of generating a new type of ETL process: loading a Data Vault Satellite table. In Data Vault, a Satellite provides context for the Core Business Concept (CBC) – embodied by the Hub and its Business Key. Or simply put: a Satellite contains...
Generating Data Vault Hubs with complex Business Keys using standard SQL
When mapping the (source) Business Key definition to the target Business Key attribute(s), the most common scenarios besides the straight-up one-to-one mapping are ‘composition’, ‘concatenation’ and ‘pivoting’. In this post I will focus on the first two, as pivoting can be implemented in different ways that (depending on the solution) do not necessarily require pattern changes.
Good news – 2nd Virtual Data Warehouse training planned for June (in the Netherlands)
I’m back again from holidays and am looking forward to continue working on the Simple Automation Series, as well as on some other new ideas that have come up during my break. I also learned that the planned training on Data Warehouse / Data Vault implementation using the Virtual Data Warehouse concept in The Netherlands (24-26 June) is nearly full. Therefore, in collaboration with Dörffler & Partner and DCYSIVE we are now planning a second...
Generating a Data Vault Hub using standard SQL
This is an article that covers how you can generate your own Hub ETL processes using standard metadata and code.
Next Virtual Data Warehouse classroom training will be in Utrecht, The Netherlands!
The upcoming Virtual Data Warehouse implementation & automation classroom training will be located in The Netherlands. Check out the details & register at www.virtualdwh.com. The training is delivered in conjunction with Doerffler & Partner and DCYSIVE and will once again focus on the design and implementation of a data platform that is truly flexible, scaleable and resilient. This is a very practical training that will provide the end-to-end overview of how to deliver a hybrid...
Applying business rules in Data Vault
As part of preparing for the Data Modelling Zone APAC 2019 I have published a new paper that covers the concepts around implementing business logic in a modern generated and/or virtualised Data Warehouse architecture. The paper can be downloaded directly here (or else in the publications section) of this website. This article covers the various architecture considerations that allow designers to bring back the concept of business logic to its core. It also makes the...
A pattern for Data Mart delivery
I’ve just completed a new article that outlines a solid approach to to deliver (generate) a Data Mart on top off a Data Vault. At least that’s what I think! I’m always open for discussions, so please contact me for questions and / or feedback. The document is the result of many discussions and tests, and really makes a case for the ‘insert-only’ Data Vault. It also covers how Data Vault can elegantly manage different...
The Virtual Data Warehouse
Over the weekend I have written up a brief overview and ‘thought piece’ of what I mean when I talk about a Virtual Data Warehouse and Data Warehouse Virtualisation. Please have a look at the article here: The Virtual Data Warehouse. A special thanks to Bret Victor for sharing a fascinating presentation on ‘inventing in principle’. This is a concept, or I should rather say a principle, I have been working on for some time now....
Schools of thought on implementing Multi-Active Satellites
Right or wrong? When it comes to data management there are almost always various alternatives for implementation and none of them are necessarily right or wrong. They represent the various options and consequences to consider, and the right solution usually is the one which is made with full understanding of these consequences, with ‘eyes wide open’. Supporting multi-active, sometimes referred to as ‘multi-variant’ or ‘multi-valued’, behaviour of Satellites is one of these areas where opinions...