Setting NB_USER for running the container should also set XDG_CACHE_HOME to that users home about docker-stacks HOT 11 CLOSED

I have no idea, but I think we can try it. XDG_CACHE_HOME was indeed added here and only for matplotlib cache: https://github.com/jupyter/docker-stacks/pull/289/files

from docker-stacks.

@benz0li could you create a PR removing two exports?

Will do.

from docker-stacks.

This should be set here (and preserved): https://github.com/jupyter/docker-stacks/blob/main/images/docker-stacks-foundation/start.sh#L125

@HasseJohansen Do you build the image yourself? Because this project does not provide GPU accelerated images.

from docker-stacks.

Hi @benz0li and thank you for the swift reply

I did notice the line in the start script setting XDG_CACHE_HOME, but somehow it doesn't work for me. I need to pass something like this when running with apptainer --env ${HOME}/cache

I actually retested without passing the above env and had the XDG_CACHE_HOME set to /home/jovyan/.cache which when using apptainer is readonly by default

I will try to reproduce with docker

I am not rebuilding the image. We get it through the singularity-hpc project (https://singularityhub.github.io/singularity-hpc/) which are using official images, but adding a GitHub hosted "recipe" for each upstream repo(in this instance Jupyter docker stacks) to build an env module(https://modules.readthedocs.io/en/latest/)

It will also convert the docker image to the .sif image format used by apptainer

As apptainer is working somewhat transparently on the host

We can do something like module load jupyter/datascience-notebook and then ex. run start-notebook.sh which is a script exposed by the env module

apptainer has an option --nv to use gpu's in the container(it will do some different things I cannot fully remember right now. They do some LD_PRELOAD stuff)

BR
Hasse

from docker-stacks.

Ok. I think I know the problem. Using the image with apptainer would newer run the start.sh/entrypoint script

I can reproduce with docker doing something like this(emulating some of stuff apptainer does by default)

docker run -it --rm --read-only -v /home/<user>/testhome:/home/<user> -v /etc/passwd:/etc/passwd:ro -e HOME=/home/<user> -e CHOWN_HOME=true --user 1000 -e NB_USER=<user> jupyter/datascience-notebook /bin/bash

<user>@7694277d45e4:~$ env|grep HOME
HOME=/home/<user>
CHOWN_HOME=true
XDG_CACHE_HOME=/home/jovyan/.cache/

<user>@7694277d45e4:~$ conda create -n torch
CondaError: Error encountered while attempting to create cache directory.
Directory: /home/jovyan/.cache/conda/notices     Exception: [Errno 30] Read-only file system: '/home/jovyan/.cache/conda'

As we don't use start.sh XDG_CACHE_HOME must come from the Dockerfile at build time (also notice that if you use Jupyter/minimal-notebook instead XDG_CACHE_HOME is not set which in my opinion is better as it will just use $HOME/.cache as default)

It comes from the scipy-notebook Dockerfile which is inherited by the datascience-notebook

from docker-stacks.

It comes from the scipy-notebook Dockerfile which is inherited by the datascience-notebook

@mathbunnyru Why not use ARG XDG_CACHE_HOME="/home/${NB_USER}/.cache/" at

from docker-stacks.

@mathbunnyru Does matplotlib strictly require XDG_CACHE_HOME to be set?

from docker-stacks.

@mathbunnyru Does matplotlib strictly require XDG_CACHE_HOME to be set?

@benz0li could you create a PR removing two exports? One in the Dockerfile and another in start.sh.
I think if it works, we should be fine.

from docker-stacks.

@HasseJohansen thanks to @benz0li we removed manually-set $XDG_CACHE_HOME.
This should fix your problem.

New images will be ready in approximately 1h30minutes.
One thing worth mentioning - we push our images to Quay.io registry, so, to receive this fix you will have to use images from there.

from docker-stacks.

Thanks. That was quick. I Think it will make the image easier to use in readonly environments. It at least fooled me that it wrote the cache in another users home when I hadn't told it to🙂

from docker-stacks.

Thanks. That was quick.

@HasseJohansen Could you please confirm that it works as expected now?

Thank you.

(I do not use apptainer and therefore cannot test)

from docker-stacks.