Category: General

Remarks, expressions and thoughts. And everything that does not fit in the other categories :)

0

Next Virtual Data Warehouse classroom training will be in Utrecht, The Netherlands!

The upcoming Virtual Data Warehouse implementation & automation classroom training will be located in The Netherlands. Check out the details & register at www.virtualdwh.de. The training is delivered in conjunction with Doerffler & Partner and will once again focus on the design and implementation of a data platform that is truly flexible, scalable and resilient. This is a very practical training that will provide the end-to-end overview of how to deliver a hybrid data warehouse...

 
0

Simple ETL generation series – an overview

I am starting a new series of articles to showcase how you can setup your own simple ETL generation / automation approach. The idea is to use the TEAM metadata for this, but it is perfectly possible to use any other source of metadata as well of course. These (working) examples should not be seen to replace commercial tools and frameworks. The posts aim to explain how ETL generation works, how it can be implemented...

 
0

Applying business rules in Data Vault

As part of preparing for the Data Modelling Zone APAC 2019 I have published a new paper that covers the concepts around implementing business logic in a modern generated and/or virtualised Data Warehouse architecture. The paper can be downloaded directly here (or else in the publications section) of this website. This article covers the various architecture considerations that allow designers to bring back the concept of business logic to its core. It also makes the...

 
0

New release of TEAM software (v1.5.4.1)

(Note: this was updated on the 22nd of February to change from 1.5.4.0 to 1.5.4.1 since this bug-fix update was released). Not too long after the previous version, we released a new update for the ‘Taxonomy of ETL Automation Metadata’ (TEAM). TEAM and VEDW are developed for the purposes of sharing knowledge and testing out new ideas. They are updated from time to time to try out new concepts and generally make it easier for...

 
0

New release of TEAM software (v1.5.3.0)

We just released a new version of the free open source front-end software to manage the essential metadata for Data Vault automation: the Taxonomy of ETL Automation Metadata (TEAM). TEAM and VEDW are developed for the purposes of sharing knowledge and testing out new ideas, but they are fully tested and complete in their functionality. You can generate your Data Vault with patterns that incorporate concepts outlined in this blog. This has been in development...

 
2

Receiving and integrating data from multiple time zones

How to capture the date / time information that is received from different international locations, across different time zones, is a challenge that comes up from time to time. Recently I was involved in some conversations about this again, which prompted me to capture this once and for all and share this here. As outlined in the pattern for Data Mart delivery you should be able to deliver information according to the timeline that the...

 
1

A pattern for Data Mart delivery

I’ve just completed a new article that outlines a solid approach to to deliver (generate) a Data Mart on top off a Data Vault. At least that’s what I think! I’m always open for discussions, so please contact me for questions and / or feedback. The document is the result of many discussions and tests, and really makes a case for the ‘insert-only’ Data Vault. It also covers how Data Vault can elegantly manage different...

 
0

The Virtual Data Warehouse

Over the weekend I have written up a brief overview and ‘thought piece’ of what I mean when I talk about a Virtual Data Warehouse and Data Warehouse Virtualisation. Please have a look at the article here:┬áThe Virtual Data Warehouse. A special thanks to Bret Victor for sharing a fascinating presentation on ‘inventing in principle’. This is a concept, or I should rather say a principle, I have been working on for some time now....

 
0

Visual Studio code editor (main screen) blank when using Biml Express – issue and fix

I encountered a very annoying issue the other day, and on the off-chance that anyone else may run into it as well I thought I’d post the fix for it. When showing some SSIS generation code using Biml Express (which is still awesome) to a colleague recently I couldn’t get the Biml files to properly open in Visual Studio using the Biml Express plug-in. The original files were still OK (not corrupted or anything), and...

 
0

Change in sample data and model availability

Hi everyone. This is just a quick note that I moved all samples from QuickDBD to SQL DBM. QuickDBD has changed its pricing model in a way that prevented me from sharing the samples in an easy way. I updated the various links on this site to reflect the new location of the data models. Sorry for any inconvenience this may have caused. The new models look pretty good I think, have a look at...