Category: General


Data Modeling Zone Europe 2014 – #DMZone

The site of the European Data Modeling Zone (DMZ) is up, and it looks really good! I look forward to being in Europe again for this event, which is hosted in Hamburg on the 29th and 30th of September 2014. On behalf of Analytics8 I will present the ‘model driven design’ approach of how the metadata that is embedded in the data model can be used to forward engineer ETL in various platforms. Feel free...


Data Vault 2.0 – how to handle Referential Integrity?

I was working on adding some of the automation code to support Data Vault 2.0 and this got me thinking about Referential Integrity (RI)  related to the modifications that Data Vault 2.0 requires. With Data Vault ‘1.0’ Referential Integrity is always enabled (except for very big systems – let’s leave that one out of the scope for now – see this older post) and in Data Vault 2.0 this hasn’t changed according to the specifications. For Data...


Data Vault 2.0 – Introduction and (technical) differences with 1.0

While finalizing the content covering Data Vault implementation it is time to start looking forward towards Data Vault 2.0. For this purpose it makes sense to provide an overview of the changes between the two data modelling approaches. To limit the scope to implementation I’ll consider it sufficient to mention that Data Vault 2.0 (DV2.0) is a complete approach covering not only the modelling (that was already part of DV1.0) but also the layered DWH...


Takeaways from the World Wide Data Vault Conference

There were some really great sessions at the World Wide Data Vault Conference (#WWDVC), and I really need to take a bit of time to upgrade some of my templates to DV2.0. Thankfully that’s pretty straightforward with the framework in place already; the changes, or upgrade, can be automated as well using the same metadata. My takeaways are; There is a push towards virtualisation for the Presentation Layer / Dimensional Model (or similar); using views directly...


User Defined Properties for ETL Automation – the final piece of the puzzle

At the end of the development efforts to support true Model Driven Design there are some elements you just don’t want to store in another metadata table. But somehow you need specific information which isn’t available in the Data Dictionary either (or directly derivable that way). In my case I wanted to label attributes within a Dimensional Model to behave following specific paradigms for presenting history – Type0, Type1 or Type2 – and essentially use this information...


Referential Integrity in Data Vault

Traditionally, Referential Integrity (RI) is not enforced for a Data Warehouse in some approaches. This may result that the Foreign Key (FK) constraint is either not generated at all, or generated but disabled on database level. The latter can improve the efficiency of the RDBMS optimiser depending on the platform used (and should by a case-by-case consideration). Unlike operational (transactional) systems, data for a DWH is prepared (scrubbed) before it is inserted and this happens in...


World Wide Data Vault Consortium (User Group) meeting

Hi everyone, I will attend the first World Wide DV Consortium (User Group) meeting in St. Albans (Vermont / USA) and talk about the Data Vault automation approaches in general, but SSIS in particular. Looking forward to it and hopefully with so many Data Vault experts we’ll get the opportunity to build & expand on this. Hope you see you there Roelant  


Comments on ‘Modeling the Agile Data Warehouse with Data Vault’

I just finished reading ‘Modeling the Agile Data Warehouse with Data Vault’ by Hans Hultgren. I think it provides a good and clear overview of the Data Warehouse design using Data Vault as technique for the core DWH layer as well and, most importantly, covers some practical aspects that have been implemented but not documented in (practical) detail such as; Key Satellites Identical Business Keys (primarily solved through defining a concatenated key) Points that are up...

Data Vault evening event – Brisbane – SSIS automation for Data Vault 0

Data Vault evening event – Brisbane – SSIS automation for Data Vault

For everyone available and interested (and in the neighbourhood) a new session has been planned for the Brisbane DWH and Data Vault modelling interest group. It’s all about using SSIS to automate your DWH development. The session is hosted on the 6th of June 2013, from 17:30 onwards. More information is here:  

Data Vault in Brisbane (Australia): the new user group is up! 0

Data Vault in Brisbane (Australia): the new user group is up!

Happy New Year! We have just initiated a platform for local Brisbane Data Vault enthusiasts to get together and share information and improve the methodology. If you’re living or working close to Brisbane, definitiely check out the Brisbane Data Vault user group. I look forward to meeting anyone interested through this new portal!