Well, it is much easier than I thought. If you know Apache, that is.
I tried to use the Live Server Plugin from Visual Code first, but that has no gzip module, so it can not be used for compressed builds. Meaning : You always would have to compile twice, and that I find stupid.
So I installed the Windows Apache Binaries from here for example, which have libz and even the Brokoli ? compression modules enabled by default.
You need to put the .htaccess from here into the Build directory of your compiled WebGL folder. Also you have to enable .htaccess overwrite in the apache config. In the file httpd.conf in the conf directory change the directive AllowOverride to from None to All for your root or project directory.
Also you should restrict the access to your apache webserver to your localhost, even if your router blocks 80/443 port, just to be on the safe side, I add this later here, for now just duckduckgo it, its easy.