Comments (18)
I haven't tried it, but I think I would opt to put all additional install steps in a new Dockerfile that derives FROM the minimal notebook like so:
FROM jupyter/minimal-notebook:4.0
/opt/conda/bin/pip install --user bash_kernel
/opt/conda/bin/python -m bash_kernel.install
I would docker build -t jupyter/bash-notebook .
and spawn a container from that image.
The start-notebook.sh script is really about late-creation of the jovyan
user and housekeeping to ensure permissions are correct for that user in the home directory and the conda directory.
from docker-stacks.
Yes, that was the initial approach I'd taken until I realized that this
wasn't working as bash_kernel was installing only for the current, i.e.
root user, so I moved it after the jovyan user creation (so out of the
Dockerfile into start-notebook.sh).
Whilst bash_kernel could install system-wide, that would solve the problem-
I think would be interesting to be able to do derivations after the
creation of the jovyan user.
I guess there were good reasons to move the jovyan user creation out of the
Dockerfile ...
On 10 September 2015 at 13:55, Peter Parente [email protected]
wrote:
I haven't tried it, but I think I would opt to put all additional install
steps in a new Dockerfile that derives FROM the minimal notebook like so:FROM jupyter/docker-stacks:4.0
/opt/conda/bin/pip install --user bash_kernel
/opt/conda/bin/python -m bash_kernel.installI would docker build -t jupyter/bash-notebook . and spawn a container
from that image.The start-notebook.sh script is really about late-creation of the jovyan
user and housekeeping to ensure permissions are correct for that user in
the home directory and the conda directory.—
Reply to this email directly or view it on GitHub
#25 (comment)
.
from docker-stacks.
Whilst bash_kernel could install system-wide, that would solve the problem-
Ah, yes, removing --user
would do the trick. I see it now.
I think would be interesting to be able to do derivations after the
creation of the jovyan user.
Do you have a use case in mind? I'm trying to think of one and have come up short (but it's still early in the day ;)
I guess there were good reasons to move the jovyan user creation out of the
Dockerfile ...
Yes. See issue #16. I've heard rumors that Docker 1.9 might finally have host/container user mapping. If so, and if it works on reasonable kernel versions, we might be able to remove the bandaid.
from docker-stacks.
Another possibility.
- Undo the user creation in script and move it back to Dockerfile so that the username / home direct exist for commands in the Dockerfile to work as expected.
- Change the start-notebook script so that instead of late-creating the user there, it instead changes in the UID/GID of the jovyan user and fixes permissions (e.g., https://muffinresearch.co.uk/linux-changing-uids-and-gids-for-user/)
Pros
- User exists at docker build time
- UID can be configured to fix host/container permission problems
Cons
- Have to scan the entire container to fix perms at start time OR limit to specific directories to scan by default (e.g., /home/jovyan, /opt/conda)
- Bandaid for docker shortcoming that may get resolved in a real-soon-now version
from docker-stacks.
I'm not aware of the uid/permission problem, but I do like the sound of
doing things cleanly in the Dockerfile, even if it requires bandaid to
workaround some hopefully temporary Docker issues.
Although my 'bash_kernel' issue is probably a feature of that kernel -
which could be fixed to install system wide, I think it would be better to
be able to install things as the target user where possible.
On 10 September 2015 at 14:38, Peter Parente [email protected]
wrote:
Another possibility.
- Undo the user creation in script and move it back to Dockerfile so
that the username / home direct exist for commands in the Dockerfile to
work as expected.- Change the start-notebook script so that instead of late-creating
the user there, it instead changes in the UID/GID of the jovyan user and
fixes permissions (e.g.,
https://muffinresearch.co.uk/linux-changing-uids-and-gids-for-user/)Pros
- User exists at docker build time
- UID can be configured to fix host/container permission problems
Cons
- Have to scan the entire container to fix perms at start time OR
limit to specific directories to scan by default (e.g., /home/jovyan,
/opt/conda)- Bandaid for docker shortcoming that may get resolved in a
real-soon-now version—
Reply to this email directly or view it on GitHub
#25 (comment)
.
from docker-stacks.
I can give the create jovyan early / fix uid later approach a shot to see if it's terribly slow or if there are other surprises.
/cc @costerwi @rgbkrk for other perspectives
from docker-stacks.
But for the moment, the workaround of doing /opt/conda/bin/pip install bash_kernel
to install it global is not that terrible. The container is for a single user anyway, imho.
from docker-stacks.
Sure ... but
- that has to be done after creation of that user
- for some reason this wasn't getting "done" on the first invocation of
start-notebook.sh (?)
I had to manually run my modified start-notebook.sh once the container
was up.
Which I can keep doing, it's just ugly and not-automated.
Thanks for the suggestions.
On 10 September 2015 at 14:52, Peter Parente [email protected]
wrote:
But for the moment, the workaround of doing /opt/conda/bin/pip install
bash_kernel to install it global is not that terrible. The container is
for a single user anyway, imho.—
Reply to this email directly or view it on GitHub
#25 (comment)
.
from docker-stacks.
- that has to be done after creation of that user
Even without the --user
flag? I thought that would work fine without the user in existence and install to /usr/local/
or some other global PREFIX.
from docker-stacks.
Oops, sorry I'll have to try that (sorry didn't know the --user flag and
assumed it did something quite different !)
Will try tonight.
On 10 September 2015 at 14:58, Peter Parente [email protected]
wrote:
- that has to be done after creation of that user
Even without the --user flag? I thought that would work fine without the
user in existence and install to /usr/local/ or some other global PREFIX.—
Reply to this email directly or view it on GitHub
#25 (comment)
.
from docker-stacks.
I just tried it and the kernel spec didn't install anywhere wtihout --user
. And now I see why: system wide install is not yet supported takluyver/bash_kernel#31.
jovyan@c0e462d94b33:/$ jupyter --paths
config:
/home/jovyan/.jupyter
/opt/conda/etc/jupyter
/usr/local/etc/jupyter
/etc/jupyter
data:
/home/jovyan/.local/share/jupyter
/opt/conda/share/jupyter
/usr/local/share/jupyter
/usr/share/jupyter
runtime:
/home/jovyan/.local/share/jupyter/runtime
from docker-stacks.
I'm not sure why the modified start-notebook.sh didn't work. Can you check docker logs
for that image?
As a different ugly workaround, a derived container may be able to install or copy misbehaving packages like this into /etc/skel
and allow the existing start-notebook.sh to move them into jovyan's home.
from docker-stacks.
With PR #31, this Dockerfile now works:
FROM jupyter/minimal-notebook
USER jovyan
# Make sure not to create a cache dir else NB_UID switching
# will hit issues.
RUN /opt/conda/bin/pip install --no-cache-dir bash_kernel
RUN /opt/conda/bin/python -m bash_kernel.install
USER root
from docker-stacks.
After the PR #31 merge,
docker pull jupyter/minimal-notebook:latest
# in a folder containing the Dockerfile from https://github.com/jupyter/docker-stacks/issues/25#issuecomment-139444841
docker build --rm -t jupyter/bash-notebook .
docker run --rm -it --p 8888:8888 jupyter/bash-notebook
Also works fine if I change the UID of jovyan (-e NB_UID=1005
).
from docker-stacks.
Thanks a lot Peter, I really appreciate the work you put in on this!
from docker-stacks.
... but ... any idea why this worked the first time (first build and run) but then I keep getting the following errors on subsequent runs?
(My host is a Fedora 22, I'm running docker 1.8.1 by the way)
2015-09-13 07:42:50,019 CRIT Supervisor running as root (no user in config file)
2015-09-13 07:42:50,019 WARN Included extra file "/etc/supervisor/conf.d/notebook.conf" during parsing
2015-09-13 07:42:50,034 INFO RPC interface 'supervisor' initialized
2015-09-13 07:42:50,034 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2015-09-13 07:42:50,034 INFO supervisord started with pid 1
2015-09-13 07:42:51,039 INFO spawned: 'notebook' with pid 9
2015-09-13 07:42:51,050 INFO exited: notebook (exit status 127; not expected)
2015-09-13 07:42:52,053 INFO spawned: 'notebook' with pid 10
2015-09-13 07:42:52,061 INFO exited: notebook (exit status 127; not expected)
2015-09-13 07:42:54,066 INFO spawned: 'notebook' with pid 11
2015-09-13 07:42:54,077 INFO exited: notebook (exit status 127; not expected)
2015-09-13 07:42:57,082 INFO spawned: 'notebook' with pid 12
2015-09-13 07:42:57,092 INFO exited: notebook (exit status 127; not expected)
2015-09-13 07:42:58,094 INFO gave up: notebook entered FATAL state, too many start retries too quickly
from docker-stacks.
OK, found the problem.
Sunday driver - I was doing a bad mount on my docker run ....
hey, it's Sunday morning ;-)
from docker-stacks.
😄
from docker-stacks.
Related Issues (20)
- Setting NB_USER for running the container should also set XDG_CACHE_HOME to that users home HOT 11
- Support for Ubuntu 23.10 version HOT 1
- The home dir should be '/root', not '/home/root', when starting the container and set the env 'NB_USER' to 'root'. HOT 2
- Conda environment not fully set in Jupyter HOT 6
- Error to start docker service using older image HOT 16
- Broken docker-stacks-foundation image HOT 1
- Broken docker-stacks-foundation image #2 HOT 2
- Install latest spark version automatically
- [jupyter/tensorflow-notebook] Kernel stays in Busy mode indefinitely HOT 6
- [BUG] Healthcheck fails when using a custom runtime dir HOT 5
- After starting the container with NB_USER=root, NB_UID=0, and NB_GID=0, $HOME environment variable is still /home/jovyan HOT 3
- Python version pinning mechanism in docker-stack-foundation does not match comment description HOT 1
- GitHub ARM64 runners are now available HOT 4
- seems broken: Infinite API requests to api/kernels/<changing-constantly>/channels?session_id=<changing-constantly> and kernel error HOT 5
- NB_USER does not have permissions to mounted directory HOT 16
- Update docker HOT 6
- Add container images for the GPU version of TensorFlow and PyTorch Notebook HOT 7
- Is the latest build correctly labeled on quay.io HOT 3
- Updates to Docker Hub have stopped HOT 3
- Default JUPYTER_PORT HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-stacks.