Why not singularity about enroot HOT 4 CLOSED

See also #1 , but the TLDR of Singularity is:

• Too complicated and huge (more than 10x enroot)
• Mostly a wrapper on top of existing cloud-native techs nowadays (might as well use runc or podman for better support)
• Not extensible
• No support for accelerators (GPUs/HCAs)
• Not a great track record (3 rewrites, several security breaches...)
• Awkward security model with setuid binaries
• Missing features (no bundles, no SLURM plugin, slow pulls, can't install packages unprivileged...)

from enroot.

Hello. One of the developers of Singularity here. I'm going to avoid arguing over the subjective points here, but I want to clarify some specific functionality that Singularity does have.

Mostly a wrapper on top of existing cloud-native techs nowadays (might as well use runc or podman for better support)

It's true we do use various OCI related libraries to support interaction docker/OCI formats etc. However the engine is independent, and Singularity has, amongst other things, its own single-file image format with embedded signing, validation, encryption etc. Users may find these advantageous.

No support for accelerators (GPUs/HCAs)

We have long had an --nv flag to support binding in of NVIDIA libraries for GPU containers, and this e.g. allows direct use of NVIDIAs dockerhub and NGC containers. We also have a corresponding --rocm flag.

Missing features (no bundles, no SLURM plugin, slow pulls, can't install packages unprivileged...)

The bundle idea is interesting. Singularity can be installed completely unprivileged, and we have a --fakeroot mode leveraging unprivileged user namespaces that allows building containers, installing packages in containers etc. without privilege. There is no SLURM plugin as running a container is as simple as running singularity run mycontainer.sif or ./mycontainer.sif like any other program in a batch script. We welcome feedback on circumstances where a plugin provides benefits.

Thanks!

from enroot.

thx for clarification!

from enroot.

It's true we do use various OCI related libraries to support interaction docker/OCI formats etc. However the engine is independent, and Singularity has, amongst other things, its own single-file image format with embedded signing, validation, encryption etc. Users may find these advantageous.

Interesting that you mention SIF, because this is actually a good example of what we didn't want. This is a non-standard format and it is not really justified. You can easily do this with plain squashfs (e.g. GPG, signify, fs-crypt, dm-crypt, etc) and you're not constrained by the ciphers or the tools that singularity offers.

We have long had an --nv flag to support binding in of NVIDIA libraries for GPU containers

While this is true, --nv is not officially supported by our container team as it doesn't rely on libnvidia-container to configure the container. As a result there are several issues that can occur while running GPU workloads in production.

Singularity can be installed completely unprivileged, and we have a --fakeroot mode leveraging unprivileged user namespaces that allows building containers, installing packages in containers etc. without privilege.

This is using setuid/setgid maps, so it is using a setuid binary and therefore is privileged.
Also setuid/setgid maps are unpractical at scale.

There is no SLURM plugin as running a container is as simple as running singularity run mycontainer.sif or ./mycontainer.sif like any other program in a batch script.

You can also do srun enroot start ... but this is very different that what the plugin does. There are a lot of considerations when integrating with SLURM (ease of use, entrypoints, MPI, image cache, etc).

from enroot.