Updated the open source Data Vault metadata tool TEAM to v188.8.131.52
We just finalised the new & improved version of the Taxonomy of ETL Automation Metadata (‘TEAM’) and published this as version v184.108.40.206. There are some major changes in this version that are not backwards compatible, so if you are using the interface views please have a look at the changes below. I have released the updated VEDW code and components as well and will do so for the ‘simple ETL generation series’ shortly.
This TEAM version is a big step towards making sure all metadata required to automate Data Vault development is in a single place and easily managed. We have generated all kinds of SQL and ETL artefacts to make sure we can cover all the scenarios we know :-).
Repository changes were needed to achieve this. The most recent version (1.5) is available here as a link to the online model. Downloads are from Github (send me a note if you would like to be added) or directly from here.
Don’t forget, there is an upcoming training on how to develop your own Virtual Data Warehouse using Data Vault techniques and (code generation as developed in TEAM) in Utrecht, The Netherlands (June 24-26). Check out the new site for this if interested: www.virtualdwh.com. Hope to see you there!
Changes in this version are:
- Updated the back-end repository to version 1. 5.
- Across-the-board name changes for repository tables and attributes (i.e. ATTRIBUTE_FROM_NAME => SATELLITE_ATTRIBUTE_NAME).
- Addition of LOAD_VECTOR to capture the inferred direction of ETL (to support connection changes). I.e. Data Vault to Data Vault or source to Data Vault.
- A snapshot of the physical model is now created during activation (MD_PHYSICAL_MODEL) and exposed as an interface (INTERFACE_PHYSICAL_MODEL). This prevents any database calls and enables a deterministic generation approach that can be integrated in DevOps.
- Various cosmetic changes (auto-resize, removing versioning where not required).
- Added support for saving interface outputs to disk as JSON files. This is another step towards becoming database-less.
- Refactor of the activation logic, to reduce code duplication via the introduction of new metadata handling classes.
- Added support for schema and database in the table names (fully qualified names). For instance adding [bdv].SAT_CUSTOMER_DERIVED.
- Various underlying changes to enable true virtualisation based on the reverse-engineered physical model. If ignore version is unchecked then everything properly runs off the internal physical model (grid).
- Minor changes to sample data, to resolve the record source when generating sample code for DIRECT.
- Fixed some issues around case-sensitive databases. The metadata should be case-sensitive, but was accidentally enclosed with an UPPER statement. Works for Azure as well now.
- Added INTERFACE_SOURCE_LINK_ATTRIBUTE_XREF interface view to expose degenerate attributes for links.
- Many tiny bugs have been squashed!
- Extended validation checks to work in true virtual mode.