Generating ETL (code) from the command line

Last week I added a new tool called ‘RunDwhAutomation’ to the Data Warehouse Automation libraries on Github.

RunDwhAutomation is a lightweight command line utility (executable) to generate code (‘ETL’) using Json files and code generation patterns as input. It doesn’t need anything else to run than the executable itself (e.g. no database, repository etc.).

The utility can ingest either a Json file, or a directory containing Json files that comform to the schema for Data Warehouse Automation. These one or more files can be applied to a template / pattern. Output can be directed to a directory with various file name and extension options, or just to the console.

RunDwhAutomation is an addition to the ecosystem for Data Warehouse Automation, complementing TEAM, TEAM_Graph, VDW and the Data Warehouse Automation class libraries. The idea is to design using the graph, manage metadata in TEAM and use VDW to create, test and manage the patterns.

RunDwhAutomation can then use the artefacts from these tools in a deployment and/or DevOps context.

I also created a brief video to explain how to generate code from the command line using RunDwhAutomation.exe with some running demos. This video can be seen on YouTube here: https://youtu.be/17UM_IOFDxU.