Multi-Arch Docker Images about fasten-onprem HOT 5 CLOSED

Thanks for all the help @jacobwgillespie @kylegalbraith !
I was able to get everything working, and fix my CI/CD pipeline as well. Depot has been working like a charm, multiarch builds are taking approx 5mins.

Once again, appreciate all your support, you've been incredibly generous 😃

from fasten-onprem.

Good call, I'll work on adding those soon.
Fasten is basically a simple Go binary, so this should be pretty easy to implement.

from fasten-onprem.

https://github.com/AnalogJ/scrutiny/blob/master/.github/workflows/docker-build.yaml

from fasten-onprem.

As discussed in the Fasten discord, here's the state of Multi-arch builds in Fasten, and a timeline of what's happened:

• when v0.0.6 was released, we were only building AMD64 docker images via Github Actions

  • front-end and backend compilation was done separately in a single multi-stage Dockerfile
  • local Docker development and "production" Docker development uses the same Dockerfile

• I attempted to add support for multi-arch builds, by uncommenting the docker/setup-qemu-action Github Action step.

  • this immediately caused build failures related to Yarn timeouts (There appears to be trouble with your network connection. Retrying...)
  • I investigated a handful of solutions to this, including yarn install --network-timeout 100000, clearing http proxies, changing the registry and clearing the cache -- none of these worked.
  • I recalled similar issues with ARM + NPM on QEMU and eventually determined that QEMU + ARM is orders of magnitude slower, and has significant issues when compiling repositories with lots of small files.

• Eventually I decided to compile the frontend app outside of Docker as a separate artifact.

  • v0.0.6...main
  • This change fixed the "frontend build", however now the backend (Go) build is failing in a similar way
  • The assumption is that this is all related to performance issues with QEMU + ARM

Possible solutions:

• moving the Go binary build outside of Docker (building multi-arch via GOOS and GOARCH), and then passing the compiled artifacts into the Dockerfile during build.
  • NEGATIVES: local development will be following a non-standard Docker development workflow, instead of running docker build, users will be required to run a make command to compile outside of docker. Build pipeline becomes more complicated
• move to custom Github Actions runners that support native (non-QEMU emulated) ARM
  • buildjet.com
  • depot.dev - (progress: conversation ongoing)
  • Azure Pipeline - (progress: only found QEMU based multi-arch build examples)

from fasten-onprem.

👋

I've opened PR #54 as an example of the Depot implementation - for some background, Depot is running managed BuildKit builders for both Intel and Arm in AWS, with a persistent 50GB SSD disk, so when build run they are able to instantly reuse the cache, no saving or loading needed.

For multi-arch builds, Depot launches both an Intel and an Arm VM and uses both, so things stay super fast and skip needing to use QEMU at all.

from fasten-onprem.