dfki-ric / docker_image_development Goto Github PK

XPRA is an remote display server and client tool that can be used as an alternative to mount x-server stuff in the exec.bash file:

Where sockets and the DISPLAY environment variable can change during the container lifetime.

A simple test (forward all desktop content) can be done like this:

In the container:

sudo apt install xpra
export XDG_RUNTIME_DIR=~/.xpra
mkdir -p $XDG_RUNTIME_DIR/xpra
xpra start --bind-tcp=0.0.0.0:10000

look for "Actual display used: :1"
export DISPLAY=:1 in the docker to forward with xpra

connext from remote:

sudo apt install xpra
xpra attach tcp:IP_WHERE_CONTAINER_RUNS:10000/

When --net=host is set in settings.bash, this can be used remotely

We could have an option in the settings.bash on what to use, and start the xpra server in the entrypont and also the client in the .exec.bash script automatically, if enabled.

The settigns.bash also needs to configure a port that is

• exposed if net=host is not used (can we expose it anyway?)
• free (perhaps check which one is still free and auto-select for local applications?)
  • port info needs to be available to be able to conntect from remote

What I would like:
to build a ROS noetic base image locally and run it with GPU support

What I did:

• edit settings.bash:
  export PROJECT_NAME=myprj
  export WORKSPACE_BASE_IMAGE=developmentimage/ros_noetic_20.04:base

./image_setup/01_base_images/build_ros_noetic_20.04.bash

• now I have:

~/ros1/docker/docker_image_development$ docker image list
REPOSITORY                                  TAG                           IMAGE ID       CREATED         SIZE
developmentimage/ros_noetic_20.04           base                          d998a5e68375   2 hours ago     3.32GB

• then I try:

 ./exec.bash devel
overriding default execmode base to: devel

using developmentimage/docker_image_development:devel

Error: No such object: developmentimage/docker_image_development:devel
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
Unable to find image 'developmentimage/docker_image_development:devel' locally
devel: Pulling from developmentimage/docker_image_development
171857c49d0f: Downloading [================================================>  ]  25.89MB/26.7MB
419640447d26: Download complete

At this point I CTRL+C to interrupt de the download. Why is the script trying to pull the image of the internet? How can I use the local base image for development (and then for release)?

Due to the username change in #35 existing devel workspaces, that are saving paths to the home directory become invalid because the new user "dockeruser" also has a new home dir (having the same uid).

This could be solved setting up the workspace from scratch or with a workaround by adding a symlink as in the previous releases:

RUN ln -s /home/dockeruser/ /home/devel
RUN chown dockeruser:dockeruser-group /home/devel

When using the workaround it could be part of the base images of individual devel images.

I'd put it into the devel image Dockerfile so every user can do it on its own, if nessecary.
Perhaps we could put it into the 02_devel_image/Dockerfile template and comment it so it is not executed by default).

I just had a use case in which I wanted the plain output of a command executed inside the container. Without all the docker and repo related output. I guess this use case could become more common and therefore I'd propose to add a -q or --quite option to the exec.bash script.

Adding more arguments to exec.bash would possibly enforce us to use proper argument handling, but I think we could keep this in mind for potential future rewrites?

In the devel image documentation is stated that the os dependencies should be added to the dockerfile and a script is provided to find out in your stack which are those.

Since autoproj update automatically installs those dependencies on its own, why adding them to the Dockerfile?

Collection of tests that need to be implemented

• exec, stop and delete for storedrelease and CD
• check adding and removing storedrelease
• assert exec, stop and delete to abort if wrong execution name is passed
• check version mismatch
• setup CD job to build devel, release and CD image => equivalent to testing image builds
• add badges to repo

Considering that --privileged effectively gives the programs inside the container root privileges on the host(by means of access to the disk and memory devices, even without /dev), are there ways to avoid that? There seem to be ways to allow only a subset of devices to be passed through(gpus, ttys).

Following case:

• Release image is build, based on "old" base image (e.g. ubuntu 16.04)
• Project moves on to a newer base image (incompatible, e.g. ubuntu 20.04)
• The extraction script from the tools folder is used to recreate a devel workspace
• The current devel image is based on the newer base image, the workspace (16.04) cannot be used with the devel image (20.04)

This should be a rare case, but has different solutions:

1. don't do those upgrades in the same project (hard to force and i think most would like to keep the project name)
2. also tag and push the devel image when building releases (shouldn't need disk space as it shares all layers with the release)
3. use the release image as devel in these cases image, but in devel mode (mount the external folders anyways)

• used more local disk space, release image cannot be deleted and contains the workspace

I guess 3. is the workaround when a tagged devel image is not available.

In the readme there is instructions for how to let your applications do rendering using the nvidia GPU: https://github.com/dfki-ric/docker_image_development/blob/master/doc/010_Setup_Docker.md#nvidia-docker-2

Right now I stumbled accross the problem that one of the things running inside the container require CUDA support from my GPU (e.g., tensorflow), which apparantly doesn't work yet, or at least requires extra steps. (Tensorflow complains about libcudart.so and lubcuda.so missing. My simulation appears in the nvidia-smi process list, but the Tensorflow processes just use the CPU instead.)

Do you have any experience with setting that up?

I found this documentation, is this how it works?

Do I need to integrate some special docker image into my image? Or build my own base image based on a nvidia/cuda image instead (normally this is used)?

Or do I need to install it inside the container as described here?

Executing the example startscript in devel mode works fine. However for release mode you get into the container first and the startscript is only executed once you try to exit the container. I noticed this in the corobx project already, but didnt find the time to look into it up until now.

I can pinpoint the location of the error, but have no idea, why it occurs. The issue is that the docker run ... line within the generate_container function in the docker_commands.bash file won't exit automatically. I already checked all passed arguments, but did not yet figure out why this is happening. I suspect it to be related to the entrypoint.sh somehow. Any idea?

When building devel/release images:

• The git repo (docker_image_development) should be tagged with the release version (date) when building a release
• The devel image should contain the base image name in the docker image description, because the selected project name does not contain the kind of OS used in the base image, making it difficult to obtain that kind of information from a registry frontend