Roelant Vos An expert view on Agile Data Warehousing

0

New release of TEAM software (v1.5.3.0)

 

 We just released a new version of the free open source front-end software to manage the essential metadata for Data Vault automation: the Taxonomy of ETL Automation Metadata (TEAM). TEAM and VEDW are developed for the purposes of sharing knowledge and testing out new ideas, but they are fully tested and complete in their functionality. You can generate your Data Vault with patterns that incorporate concepts outlined in this blog. This has been in development...

2

Receiving and integrating data from multiple time zones

 

  How to capture the date / time information that is received from different international locations, across different time zones, is a challenge that comes up from time to time. Recently I was involved in some conversations about this again, which prompted me to capture this once and for all and share this here. As outlined in the pattern for Data Mart delivery you should be able to deliver information according to the timeline that...

1

A pattern for Data Mart delivery

 

 I’ve just completed a new article that outlines a solid approach to to deliver (generate) a Data Mart on top off a Data Vault. At least that’s what I think! I’m always open for discussions, so please contact me for questions and / or feedback. The document is the result of many discussions and tests, and really makes a case for the ‘insert-only’ Data Vault. It also covers how Data Vault can elegantly manage different...

0

The Virtual Data Warehouse

 

 Over the weekend I have written up a brief overview and ‘thought piece’ of what I mean when I talk about a Virtual Data Warehouse and Data Warehouse Virtualisation. Please have a look at the article here: The Virtual Data Warehouse. A special thanks to Bret Victor for sharing a fascinating presentation on ‘inventing in principle’. This is a concept, or I should rather say a principle, I have been working on for some time now....

0

Visual Studio code editor (main screen) blank when using Biml Express – issue and fix

 

 I encountered a very annoying issue the other day, and on the off-chance that anyone else may run into it as well I thought I’d post the fix for it. When showing some SSIS generation code using Biml Express (which is still awesome) to a colleague recently I couldn’t get the Biml files to properly open in Visual Studio using the Biml Express plug-in. The original files were still OK (not corrupted or anything), and...

0

Change in sample data and model availability

 

 Hi everyone. This is just a quick note that I moved all samples from QuickDBD to SQL DBM. QuickDBD has changed its pricing model in a way that prevented me from sharing the samples in an easy way. I updated the various links on this site to reflect the new location of the data models. Sorry for any inconvenience this may have caused. The new models look pretty good I think, have a look at...

0

Early-bird for upcoming training (22-24 October) about to expire

 

 If you are interested in understanding and discussing the intricacies of developing a Data Vault based data platform / Data Warehouse, then please consider the upcoming Data Vault implementation training course. It is a practical training focused on understanding the impacts of various design decisions on the (Data Vault) patterns and overall solution architecture. Regardless whether you develop your solution using custom scripting or by using an ‘off-the-shelf’ vendor application, you need detailed understand how...

0

How to be sure you load 100% of the data 100% of the time!

 

 How can you be sure you load 100% of the data 100% of the time? This is an article that expands on a topic that often comes up in the Data Vault implementation training: applying Referential Integrity and consistency checks in Data Vault. It sounds straightforward, but in reality these are interesting topics for any Data Warehouse (DWH) and Data Vault practitioner. I started to write a blog post about it, but before I knew...

3

Schools of thought on implementing Multi-Active Satellites

 

 Right or wrong? When it comes to data management there are almost always various alternatives for implementation and none of them are necessarily right or wrong. They represent the various options and consequences to consider, and the right solution usually is the one which is made with full understanding of these consequences, with ‘eyes wide open’. Supporting multi-active, sometimes referred to as ‘multi-variant’ or ‘multi-valued’, behaviour of Satellites is one of these areas where opinions...

2

Update on development & collaboration efforts

 

 Development update This is a status update for the small community of open source developers and collaborators that use, or contribute to, the growing ecosystem of metadata management and Data Vault generation tooling. I’ve worked hard on creating a more stable set of tooling around ETL generation and as a result the TEAM and VEDW applications (as well as some others) have now been completely separated with a view to increase greater interoperability with other...