[Bazel] Dynamically generate Emscripten cache about emsdk HOT 5 CLOSED

One alternative long term solution might be build all the system libraries from source via bazel rules. I know @walkingeyerobot has been thinking about this for a while now, but I'm not sure how close it is.

from emsdk.

I am not necessarily against that approach but that would take a bit of time, right? From (thin) LTO, PIC, and 64-bit, we have 16 (?) possible combinations multiplied by around 40 different ports/libraries? That's 640 jobs based on my estimates...

from emsdk.

I am not necessarily against that approach but that would take a bit of time, right? From (thin) LTO, PIC, and 64-bit, we have 16 (?) possible combinations multiplied by around 40 different ports/libraries? That's 640 jobs based on my estimates...

The idea is that those system libraries would be just like any other sources in your project so only the precise config that you need to for a given build would be produced. There would be no need to enumerate them or build all of them ever.

from emsdk.

I am fine with this change in general. I'm too far from getting runtimes on demand to work with bazel, and if you have the bandwidth and motivation to contribute this, I'm happy to accept it.

However, I don't think we can make use of bash in genrules. The bazel toolchain works on linux, mac, and windows, and while the bash that you have will work great on linux, bash on mac has some odd quirks and on windows it simply doesn't exist. If you can write this in something more portable (i.e. python) then I think that'll be fine.

Also, this is complex enough that I would ask for a smoketest to be written for CI for all three supported platforms.

from emsdk.

I am fine with this change in general. I'm too far from getting runtimes on demand to work with bazel, and if you have the bandwidth and motivation to contribute this, I'm happy to accept it.

However, I don't think we can make use of bash in genrules. The bazel toolchain works on linux, mac, and windows, and while the bash that you have will work great on linux, bash on mac has some odd quirks and on windows it simply doesn't exist. If you can write this in something more portable (i.e. python) then I think that'll be fine.

Also, this is complex enough that I would ask for a smoketest to be written for CI for all three supported platforms.

I think it can be easily wrapped into a Python script, after all, it is mostly just running embuilder.py anyway.

from emsdk.