The Virtual Data Warehouse
Over the weekend I have written up a brief overview and ‘thought piece’ of what I mean when I talk about a Virtual Data Warehouse and Data Warehouse Virtualisation.
This is a concept, or I should rather say a principle, I have been working on for some time now. The first time I presented an early version of this idea was at the 2nd Worldwide Data Vault Conference hosted by Dan Linstedt a few years back, where I was working on a Data Warehouse without having access to any database server at the time :-).
Around this time I also started developing on the ‘Virtual Enterprise Data Warehouse’ software (available on Github, or check the collaboration page) as a mechanism for ongoing exploration of these concepts and the possibilities they open up. I never had any ideas to commercialise this software, and I don’t have plans for this now (and in the future). However, continuing working on this still is a great way to ‘keep the mind sharp’ and generally try out new ideas.
Although I appreciate it is easy to say, I don’t worry about performance too much. This is about defining the principles, the mindset, to make sure the direction is set for further development on better ways to expose data to consumers.
In a practical sense, performance is always a consideration of course. So far though, it has been proven to be perfectly doable to ‘toggle’ between virtual and persistent deployment of a Data Warehouse solution.