Category: General

Data Vault implementation A-Z: Staging data (the conceptual side) 0

Data Vault implementation A-Z: Staging data (the conceptual side)

This is the first of a planned series of implementation designs for implementing Data Vault in an end-to-end Data Warehouse environment. The positioning of the Data Vault concepts and techniques in the greater design of the system (reference architecture) is documented elsewhere in this site, mainly in the ‘papers’ section. Data Vault in itself does not provide a complete solution for most Data Warehouse purposes but provides a great set of modeling techniques to design the...

Data Vault comparisons 1

Data Vault comparisons

I have drafted a comparison between Data Vault and normalised (3NF) and denormalised (Kimball) models for reference. This comparison is applicable for using these models as the core Data Warehouse model as opposed to modelling for reporting purposes (i.e. data marts).  

Data Vault versus the persistent Staging Area 3

Data Vault versus the persistent Staging Area

One of the questions I regularly get during presentations is what the benefits of Data Vault are over a persistent Staging Area. In other words: why go through the effort of defining a Data Vault model when you can receive the same ‘regeneration / recreation’ capabilities with a persistent Staging Area which directly feeds a Dimensional Model or similar presentation model. First off; in my reference architecture I use both the Data Vault (in the...

Changes to the site(s) 0

Changes to the site(s)

In preparation of enabling an online (proof of concept) ETL generation site  I have updated various sections of this weblog to conform to the implementations (templates) used for ETL generation. Moreover most documentation has been moved to the corresponding Wiki and consequently removed from this site. Ultimately the Wiki is better suited to add more detailed documentation and examples and as such only the high level concepts and positioning are available on this site. To complete the...

Data Vault in Australia 0

Data Vault in Australia

Anyone interested in Data Vault in Australia, please sign up for the official Data Vault User Groups (and specifically the Australian group). Just started, and Data Vault does not have many reference projects in Australia, but that can only mean there is much opportunity to set up a good foundation. The site can be found on EDIT: the user groups have been dismantled in favour of the linkedIN discussion groups!!!  

Enterprise Architecture seminar thoughts 0

Enterprise Architecture seminar thoughts

This posting is about the most complex object that humankind can come up with; an Enterprise. At least that is what John Zachman states at his one day seminar about Enterprise Architecture (physics 101 – Monday 23rd of August 2010 – Sydney). The first rule for the way of working is defined right at the start: when doing anything with an enterprise or start designing anything for an enterprise; make sure you understand the enterprise...

Completed CBIP certification 3

Completed CBIP certification

As of July 2010, Roelant Vos is a certified business intelligence professional (CBIP), specialized in data management! The CBIP exam involves a minimum of three examinations ranging from generic IT to datawarehousing and data modelling. I’m very happy that my employer Analytics8 made this possible.  

Why you need an Enterprise BI framework 0

Why you need an Enterprise BI framework

Over the last few months I have done a number of pre-sales presentations regarding a framework for designing and developing data integration programs. While the framework in question is a smaller scale (mainly ETL) I did encounter numerous discussion why such a framework is necessary in the first place. What would be achieved by using it? Because this blog serves as my own personal framework and collection of best practices I thought it would be...

Current mapping generation improvement points 0

Current mapping generation improvement points

I’ve demo-ed the posted processes for mapping generation and Data Vault a couple of times for varying audiences and (lucky me) no one has noticed the existing flaws 🙂 So it’s time to make a list of them to keep me from not forgetting to fix them. And it’s a heads up for the few people I know who are using the scripts. There is currently no way to keep the ‘transactional’ attributes out of...