Code Monkey home page Code Monkey logo

gpu-jupyterhub's Introduction

This repo contains the docker-compose.yml for defining the dependencies to run the Jupyter Notebook Images described in the Dockerfiles under Jupyter_Images. There is one Jupyter Notebook image, CUDA10.0 and Ubuntu18.04 based. It will be spwaned from a Jupyterhub which is defined in the Dockerfile.jupyterhub under the folder Jupyterhub_Image. For more details visit our website https://www.dlm.med.fau.de/setting-jupyterhub-deep-learning/

Edited the repo:

  • removed the cuda 9 image
  • restructured the docker-compose.yml --> you will need a folder called secets containing the env_files mentioned in the yml-env_file --> will need a folder under Jupyterhub_Image called ssl containing your ssl certs for building and running the Jupyterhub.

The Jupyter_Image (jupyternotebook) has new features and some removed features: Python2 is gone R is available Python3 is available SOS is available

The workhorse for deep-learning is Python3 based. It contains Tensorflow-GPU 1.14, Keras 2.2.4 Pytorch 1.2.0 and Fastai 1.0.57

-Jupyterhub Image definition with DockerSpawner for spawning Jupyternotebooks. Localauthentication is removed and replaced by OAuth with Github

  • Data is persisted: --> -locally (Docker-Volume --> cookie secrets) --> host machine and pesronal data is mapped into the container for pre spwan hook (see into jupyterhub_config.py)

-Spawned Images run in single Docker-Containers
- Data is persisted: --> user based: -locally on nvme/ssd (Docker-Volume bind mount--> host_path:/home/Deep_Learner/private/local) and per network associated folder
(host_path:/home/Deep_Learner/private/network). --> commonly shared: - per network associated folder (net_share:/home/Deep_Learner/shared).

Added ftp server mapping to an uploads folder in host_path/uploads:/home/Deep_Learner/shared/uploads

gpu-jupyterhub's People

Contributors

samirjabari avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

gpu-jupyterhub's Issues

swarm gpu support in docker stack/compose

Hello I saw your docker-compose file
As I understanded It's compatbile with docker-swarm, docker-stack, nvidia-gpu
Could you please explain me how to allocate GPU to a container with a docker-compose file for docker-swarm deployment

PS : sorry for my bad english

tornado.httpclient.HTTPClientError: HTTP 403: Forbidden

I build the image but have run into this issue. I have been trying get this to work but no matter with SSL/no SSL/different ports/checking for a running configureable proxy etc has not helped. Hoping you might throw some light on this issue with this.

jupyterhub | [I 2018-09-21 16:45:06.198 JupyterHub app:1667] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.3
jupyterhub | [I 2018-09-21 16:45:06.198 JupyterHub app:1667] Using Spawner: dockerspawner.dockerspawner.DockerSpawner-0.9.1
jupyterhub | [I 2018-09-21 16:45:06.204 JupyterHub app:1053] Writing cookie_secret to /data/jupyterhub_cookie_secret
jupyterhub | [I 2018-09-21 16:45:06.236 alembic.runtime.migration migration:117] Context impl SQLiteImpl.
jupyterhub | [I 2018-09-21 16:45:06.237 alembic.runtime.migration migration:122] Will assume non-transactional DDL.
jupyterhub | [I 2018-09-21 16:45:06.258 alembic.runtime.migration migration:327] Running stamp_revision -> 896818069c98
jupyterhub | [I 2018-09-21 16:45:06.575 JupyterHub auth:547] Creating user: adduser -q --gecos '""' --disabled-password admin
jupyterhub | [I 2018-09-21 16:45:06.700 JupyterHub auth:547] Creating user: adduser -q --gecos '""' --disabled-password user
jupyterhub | [I 2018-09-21 16:45:06.817 JupyterHub app:1849] Hub API listening on http://jupyterhub:5264/hub/
jupyterhub | [I 2018-09-21 16:45:06.818 JupyterHub proxy:567] Starting proxy @ http://:443/
jupyterhub | 16:45:07.228 - info: [ConfigProxy] Proxying https://*:443 to (no default)
jupyterhub | 16:45:07.229 - info: [ConfigProxy] Proxy API at http://127.0.0.1:8001/api/routes
jupyterhub | 16:45:07.371 - warn: [ConfigProxy] 403 GET /api/routes
jupyterhub | [E 2018-09-21 16:45:07.371 JupyterHub app:1952]
jupyterhub | Traceback (most recent call last):
jupyterhub | File "/opt/conda/lib/python3.6/site-packages/jupyterhub/app.py", line 1950, in launch_instance_async
jupyterhub | await self.start()
jupyterhub | File "/opt/conda/lib/python3.6/site-packages/jupyterhub/app.py", line 1895, in start
jupyterhub | await self.proxy.check_routes(self.users, self._service_map)
jupyterhub | File "/opt/conda/lib/python3.6/site-packages/jupyterhub/proxy.py", line 57, in locked_method
jupyterhub | return await method(*args, **kwargs)
jupyterhub | File "/opt/conda/lib/python3.6/site-packages/jupyterhub/proxy.py", line 297, in check_routes
jupyterhub | routes = await self.get_all_routes()
jupyterhub | File "/opt/conda/lib/python3.6/site-packages/jupyterhub/proxy.py", line 733, in get_all_routes
jupyterhub | resp = await self.api_request('', client=client)
jupyterhub | File "/opt/conda/lib/python3.6/site-packages/jupyterhub/proxy.py", line 694, in api_request
jupyterhub | result = await client.fetch(req)
jupyterhub | tornado.httpclient.HTTPClientError: HTTP 403: Forbidden

Password for Deep_Learner

Hi,

After installation, I'm trying to some conda updates, however this is not permitted without Deep_Learner sudo priviledges. Where is the password for this user determined in the code, and is the password the same for all user containers?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.