This post details how you can connect your own Biml automation framework to the schema for Data Warehouse Automation. A focus on being technology agnostic, and repository-less The schema for Data Warehouse Automation provides a way to separate the storage of your design and code generation (meta) data from how you interact with it and how you apply it to generate data solutions. This (meta)data is something I refer to as design metadata, and it...
Introduction – back in code generation mode Things have been busy after joining Varigence a few months ago. A major focus has been to develop new code generation features, to enable the BimlFlex data warehouse automation platform to use Azure Data Factory (ADF) Mapping Data Flows (‘Data Flows’)- in particular to interfacing with Delta Lake storage. Basically, to make sure that we can use BimlScript to generate Delta Lake compliant Data Flows and integrate this...
One of the things to ‘solve’ working for a software vendor is how to balance delivering meaningful content for the community for collaboration purposes with commercial software development and sales. We’re aligned that we want to make this work, so are continuing to discuss how we can do both. The first action on this is the creation of a public Github repository that contains community-facing content that may be useful working with the Varigence toolset....
After many years I have finished up at Allianz to join the Varigence team, so that I can work on the BimlFlex solution for data solution automation. Automating data solution has always been my long-term interest, and working for/with Varigence is an amazing opportunity to contribute to what is already a great suite of tooling. We’ll incorporate many ideas that are still ‘on the table’, and won’t lose sight of the open source community. Working...
