rootless-containers / rootlesskit Goto Github PK

View Code? Open in Web Editor NEW

926.0 926.0 93.0 3.61 MB

Linux-native "fake root" for implementing rootless containers

License: Apache License 2.0

Go 90.52% Shell 6.92% Dockerfile 2.18% Makefile 0.37%

rootless-containers

rootlesskit's People

Contributors

Stargazers

Watchers

Forkers

akihirosuda tonistiigi shaneutt tiborvass westonsteimel u5surf mikefedyk seemethere larsfronius jaredbancroft eschen42 simhaonline zyqsempai ibuildthecloud khyveasna11111908 kolyshkin tylarb pbi-46 akhilerm zhsj thecompuguru artbegolli ryotarai jasonyangshadow j11332 happytobi salewski rhatdan njc-gov jaydoubleu giuseppe iceber thajeztah iamthatiam777 mukesh23singh timyinshi narutogit djs55 bakoorahnin ilgrim melyseengit ekmixon luap99 https-github-com-feeflyons-cards-llc alanyoshida amulmgr joel113 lolalset drdivano pdbrown prudhvirajtirumanisetti manugupt1 wormon oneflyingfish fwilhe-containers kubegc hlinbo mrcodechef saschaschwarze0 hown3d kowalski7cc nibaozhu8 duguhaotian sitedata zhangwenlong8911 mc-etch desenna g1eng raharper venuvasuu21 fahedouch iq-scm i-do-cpp karahiyo x-oss-byte evernodexrpl rex2501 mmcachran loongson-cloud-community kafnevod wolfi-chainguard-demo nawras2002 mirantis kkpan11 antrusd j-mata chainguard-wolfi-bites-back devopseng99 xanimo

rootlesskit's Issues

v0.9.2 signal forwarder doesn't support --pidns

#127 doesn't support --pidns.

Without --pidns (works as expected)

$ dockerd-rootless.sh --experimental
...
INFO[2020-03-15T11:02:42.263959577+09:00] API listen on /run/user/1001/docker.sock     
^CINFO[2020-03-15T11:02:43.493624366+09:00] Processing signal 'interrupt'                
INFO[2020-03-15T11:02:43.494655209+09:00] Processing signal 'interrupt'                
INFO[2020-03-15T11:02:43.495386457+09:00] stopping event stream following graceful shutdown  error="<nil>" module=libcontainerd namespace=moby
INFO[2020-03-15T11:02:43.495546067+09:00] Daemon shutdown complete                     
INFO[2020-03-15T11:02:43.495566660+09:00] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
INFO[2020-03-15T11:02:43.495587614+09:00] stopping healthcheck following graceful shutdown  module=libcontainerd
INFO[2020-03-15T11:02:43.495986508+09:00] Processing signal 'interrupt'
$ echo $?
0

With --pidns (unexpected failure)

$ DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS=--pidns dockerd-rootless.sh --experimental
...
INFO[2020-03-15T13:28:53.859447390+09:00] API listen on /run/user/1001/docker.sock     
^CINFO[2020-03-15T13:28:54.892282069+09:00] Processing signal 'interrupt'                
INFO[2020-03-15T13:28:54.892349636+09:00] Processing signal 'interrupt'                
INFO[2020-03-15T13:28:54.895970152+09:00] stopping event stream following graceful shutdown  error="<nil>" module=libcontainerd namespace=moby
INFO[2020-03-15T13:28:54.898414592+09:00] Daemon shutdown complete                     
INFO[2020-03-15T13:28:54.898938906+09:00] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
INFO[2020-03-15T13:28:54.899736172+09:00] stopping healthcheck following graceful shutdown  module=libcontainerd
[rootlesskit:child ] error: command [/usr/local/bin/dockerd-rootless.sh --experimental] exited: waitid: no child processes
[rootlesskit:parent] error: child exited: exit status 1
$ echo $?
1

`ip` command fails on alpine in a container

I'm trying to get rootless docker-in-docker working and I've managed to get it installed and dependencies set up but the ip link set command used as part of setting up the networking (I think by rootlesskit) fails as if it was given invalid arguments:

/ $ dockerd-rootless.sh --experimental --storage-driver vfs
+ '[' -w /tmp/docker-1000 ]
+ '[' -w /home/user ]
+ rootlesskit=
+ which docker-rootlesskit
+ which rootlesskit
+ rootlesskit=rootlesskit
+ break
+ '[' -z rootlesskit ]
+ : 
+ : 
+ net=
+ mtu=
+ '[' -z ]
+ which slirp4netns
+ slirp4netns --help
+ grep -- --disable-host-loopback
--disable-host-loopback  prohibit connecting to 127.0.0.1:* on the host namespace
+ net=slirp4netns
+ '[' -z ]
+ mtu=65520
+ '[' -z slirp4netns ]
+ '[' -z 65520 ]
+ '[' -z ]
+ _DOCKERD_ROOTLESS_CHILD=1
+ export _DOCKERD_ROOTLESS_CHILD
+ exec rootlesskit '--net=slirp4netns' '--mtu=65520' --disable-host-loopback '--port-driver=builtin' '--copy-up=/etc' '--copy-up=/run' /home/user/bin/dockerd-rootless.sh --experimental --storage-driver vfs
WARN[0000] "builtin" port driver is experimental        
BusyBox v1.30.1 (2019-06-12 17:51:55 UTC) multi-call binary.

Usage: ip [OPTIONS] address|route|link|tunnel|neigh|rule [ARGS]

OPTIONS := -f[amily] inet|inet6|link | -o[neline]

ip addr add|del IFADDR dev IFACE | show|flush [dev IFACE] [to PREFIX]
ip route list|flush|add|del|change|append|replace|test ROUTE
ip link set IFACE [up|down] [arp on|off] [multicast on|off]
	[promisc on|off] [mtu NUM] [name NAME] [qlen NUM] [address MAC]
	[master IFACE | nomaster]
ip tunnel add|change|del|show [NAME]
	[mode ipip|gre|sit] [remote ADDR] [local ADDR] [ttl TTL]
ip neigh show|flush [to PREFIX] [dev DEV] [nud STATE]
ip rule [list] | add|del SELECTOR ACTION
[rootlesskit:parent] error: failed to setup network &{binary:slirp4netns mtu:65520 ipnet:<nil> disableHostLoopback:true apiSocketPath:}: setting up tap tap0: executing [[nsenter -t 16 -n -m -U --preserve-credentials ip tuntap add name tap0 mode tap] [nsenter -t 16 -n -m -U --preserve-credentials ip link set tap0 up]]: exit status 1
[rootlesskit:child ] error: parsing message from fd 3: EOF

If I run the ip link command manually it runs in a more expected way (the interface doesn't exist because slirpnet4ns hasn't set it up).

I'm running this in a fairly weird environment - inside a docker container (obviously privileged but I'm hoping it can be restricted specifically to seccomp/apparmor/unmasked sys/proc mounts):

FROM golang:1.11-alpine AS gobuild-base
RUN apk add --no-cache \
	bash \
	build-base \
	gcc \
	git \
	libseccomp-dev \
	linux-headers \
	make

FROM alpine:3.8 AS idmap
RUN apk add --no-cache autoconf automake build-base byacc gettext gettext-dev gcc git libcap-dev libtool libxslt
RUN git clone https://github.com/shadow-maint/shadow.git /shadow
WORKDIR /shadow
RUN git checkout 59c2dabb264ef7b3137f5edb52c0b31d5af0cf76
RUN ./autogen.sh --disable-nls --disable-man --without-audit --without-selinux --without-acl --without-attr --without-tcb --without-nscd \
  && make \
  && cp src/newuidmap src/newgidmap /usr/bin

FROM gobuild-base AS rootlesskit
RUN go get github.com/rootless-containers/rootlesskit/cmd/rootlesskit && go get github.com/rootless-containers/rootlesskit/cmd/rootlessctl

FROM gobuild-base AS slirp4netns
RUN apk add --no-cache autoconf automake glib-dev glib-static
RUN git clone https://github.com/rootless-containers/slirp4netns.git /slirp4netns
WORKDIR /slirp4netns
RUN ./autogen.sh \
  && LDFLAGS=-static ./configure --prefix=/usr \
  && make \
  && make install

FROM alpine:3.10
COPY --from=idmap /usr/bin/newuidmap /usr/bin/newuidmap
COPY --from=idmap /usr/bin/newgidmap /usr/bin/newgidmap
COPY --from=rootlesskit /go/bin/rootlesskit /usr/bin/rootlesskit
COPY --from=rootlesskit /go/bin/rootlessctl /usr/bin/rootlessctl
COPY --from=slirp4netns /slirp4netns/slirp4netns /usr/bin/slirp4netns
RUN chmod u+s /usr/bin/newuidmap /usr/bin/newgidmap \
  && adduser -D -u 1000 user \
  && mkdir -p /run/user/1000 \
  && chown -R user /run/user/1000 /home/user \
  && echo user:100000:65536 | tee /etc/subuid | tee /etc/subgid
RUN apk add --no-cache curl iptables
USER user
RUN curl -sSL https://get.docker.com/rootless | sh 
ENV XDG_RUNTIME_DIR=/tmp/docker-1000
ENV PATH=/home/user/bin:$PATH
ENV DOCKER_HOST=unix:///tmp/docker-1000/docker.sock

--copy-up=/etc does not work on CentOS 7

[suda@suda-centos ~]$ rootlesskit --copy-up=/etc bash
WARN[0000] specifying --disable-host-loopback is highly recommended to prohibit connecting to 127.0.0.1:* on the host namespace (requires slirp4netns v0.3.0+ or VPNKit)
[rootlesskit:child ] error: symlinking /proc/self/mounts to /etc/mtab: symlink /proc/self/mounts /etc/mtab: file exists
[rootlesskit:parent] error: child exited: exit status 1
[suda@suda-centos ~]$ ls -l /etc/mtab
lrwxrwxrwx. 1 root root 17 Jan 16 21:27 /etc/mtab -> /proc/self/mounts
[suda@suda-centos ~]$ rootlesskit -v
rootlesskit version 0.3.0-alpha.0

Initially reported by @tonistiigi (thanks!) tonistiigi/docker-install#1 (comment)

remove dependency on github.com/docker/docker/pkg/system

/etc/resolv.conf is unmounted when recreated on the initial namespace

On Fedora 28, /etc/resolv.conf is a regular file (unlike Ubuntu, see #4) and often recreated by NetworkManager.

Bind-mounted /etc/resolv.conf in the userns is unexpectedly unmounted when /etc/resolv.conf in the initial ns is recreated by NetworkManager.

Workaround ideas:

watch re-creation event using inotify, and redo mount on re-creation?
chroot to a minimal Linux distro rootfs (e.g. alpine) and bind-mount $HOME and $XDG_RUNTIME_DIR? (This would be more reliable but ideally I don't want to introduce extra chroot..)

intermittent api.sock: bind: address already in use

+ rootlesskit --state-dir /run/user/1001/usernetes/rootlesskit --net=slirp4netns --mtu=65520 --copy-up=/etc --copy-up=/run --copy-up=/var/lib ./run.sh default-crio
[rootlesskit:parent] error: listen unix /run/user/1001/usernetes/rootlesskit/api.sock: bind: address already in use

port back errCh fix libpod#5183

containers/podman#5183

Running all containers with a single privileged user

From https://github.com/containers/libpod/blob/master/README.md

Rootless Podman runs locked-down containers with no privileges that the user running the container does not have. Some of these restrictions can be lifted (via --privileged, for example), but rootless containers will never have more privileges than the user that launched them.

I assume the same is true for rootless Docker.

With that in mind, would it be a bad idea to run all your services inside containers using the root user? Maybe Single-mapping mode considered that. The problems I'm facing are mostly related to volume and folder permissions/IDs between host and containers. And the requirements of builds and each service of course.

It's not possible to make a non-root ID match e.g. 1000:1000 from the host because

Inside rootlesskit, your UID is mapped to 0 but it is not the real root

and the volumes are owned by root.

support primary group name != user name

vagrant@vagrant-openSUSE-Tumbleweed:~/usernetes> id
uid=1000(vagrant) gid=100(users) groups=100(users),1000(vagrant)
vagrant@vagrant-openSUSE-Tumbleweed:~/usernetes> cat /etc/subuid 
vagrant:231072:65536
vagrant@vagrant-openSUSE-Tumbleweed:~/usernetes> cat /etc/subgid
vagrant:231072:65536
vagrant@vagrant-openSUSE-Tumbleweed:~/usernetes> ./bin/rootlesskit bash
vagrant-openSUSE-Tumbleweed:~/usernetes # cat /proc/self/uid_map 
         0       1000          1
         1     231072      65536
vagrant-openSUSE-Tumbleweed:~/usernetes # cat /proc/self/gid_map 
         0        100          1

vpnkit performance

I tried to measure network speed when using vpnkit with rootlesskit. One thing I noticed was that the upload speed was very low by default, like 200x lower than expected. After tweaking https://github.com/AkihiroSuda/rootlesskit/blob/c166fc61352542ac0423b5abe84d00450f2b7197/pkg/child/child.go#L154 I could make it much more performant and actually faster than the download speed. I'm not sure how reproducible that is and what the correct value should be so didn't make a PR yet. Trying different values from 1500 to 32K it appeared to max somewhere around the middle for me but my measurement method wasn't very precise.

There may be some other things to try with a better benchmark. With the tweaks, it still was ~1.5-2x slower than baseline in my case. And slower than Docker for Mac that also uses vpnkit (although a different os).

port: builtin: UDP reply packet is dropped

Terminal 1:

$ rootlesskit --net=slirp4netns --port-driver=builtin -p 0.0.0.0:1234:1234/udp  -- sh -c "nc -u -l -p 1234 < /etc/issue"

Terminal 2:

$ echo foo | nc -u 127.0.0.1 1234

foo is printed on Terminal 1, but /etc/issue is not printed on Terminal 2 when --port-driver is set to builtin.

--port-driver=socat works as expected.

version: e6152ff

containers/podman#4586 (comment)

proxy SIGINT

When dockerd receives SIGINT (Ctrl-C), dockerd handles the signal and gracefully shuts down with exit code 0.

This currently doesn't work with dockerd-rootless.sh.

Probably this function can be used for proxying signals: https://github.com/docker/cli/blob/7f612bfca67119c61461b61dd37b998668937f07/cli/command/container/tty.go#L100

builtin port forwarder sometimes got stuck in "Connection reset by peer"

Reported in containers/podman#7016

Reproducible with both Docker and Podman with podman run -d -p 8080:80 nginx:alpine, while true; do curl -L http://127.0.0.1:8080; done

Support deterministic MAC address

Support specifying --mac=01:23:45:67:89:AB
By default, (the lower 24 bits of?) the MAC should be computed from the hash of --state-dir?

ValidatePortSpec should allow duplicated children

containers/podman#5646 (comment)

/etc/subuid: support numeric IDs

moby/moby#40926

Propose `securityContext.sysMount: Unmasked` to Kubernetes

It turned out that running rootlesskit --net=slirp4netns inside container requires unmasked sysfs, so as to mount sysfs with expected /sys/class/net entries.

#23 (comment)

support launching slirp4netns with --disable-host-loopback

workaround for rootless-containers/slirp4netns#11

support slirpnetstack (as a binary & as a Go pkg)

https://github.com/majek/slirpnetstack

Possibility to connect to a VPN for the network NS created by rootlesskit

I would like to have per process VPN available. Is it possible from rootlesskit to connect to a VPN and have it available in the process spawned by rootlesskit. Is it as simple as connecting to the VPN inside the shell created by rootlesskit

redirect slirp4netns stdout/stderr to logrus.Debug

Support new pid namespace or ability to kill all children

This is a follow up to our discussion at DockerCon.

Rootlesskit (and the usernetes work) has been integrated it k3s but results in several usability issues. For now it seems the simplest approach to making rootless with k3s more user friendly would be if rootkit could ensure some way that if the children of executed process died when the process died. This could be done by creating a new pid namespace or possibly with cgroup (although I don't know if rootless can modify cgroups).

Remove vdeplug_slirp (already deprecated since v0.4.0)

vdeplug_slirp has been deprecated (#62) since v0.4.0, and will be removed in v0.9.0.

Should the second `rootlesskit` instance ns-enter the first one? Or should it just create different NS?

remove dependency on `mount` binary

https://github.com/rootless-containers/rootlesskit/blob/master/pkg/child/child.go has extra exec for mount(8) binary.

mount(2) syscall could be called directly.

TODO: add parent-child IPC

Especially, the child needs to wait for the parent's newuidmap/newgidmap completion.

lxc-user-nic should support DHCP Renew

panic: expected to retain 10.0.3.206, got 10.0.3.207

goroutine 6 [running]:
github.com/rootless-containers/rootlesskit/pkg/network/lxcusernic.dhcpRenewRoutine(0xc000190fc0, 0xc000029080, 0x4, 0xc000029a2c, 0x4, 0x4, 0x34630b8a000)
        /home/suda/gopath/src/github.com/rootless-containers/rootlesskit/pkg/network/lxcusernic/lxcusernic.go:147 +0x2d3
created by github.com/rootless-containers/rootlesskit/pkg/network/lxcusernic.(*childDriver).ConfigureNetworkChild
        /home/suda/gopath/src/github.com/rootless-containers/rootlesskit/pkg/network/lxcusernic/lxcusernic.go:129 +0x82a
[rootlesskit:parent] error: child exited: exit status 2

https://github.com/rootless-containers/rootlesskit/blob/v0.9.4/pkg/network/lxcusernic/lxcusernic.go#L133-L151

Support disabling specific drivers using go build tags

https://salsa.debian.org/go-team/packages/rootlesskit/-/blob/74020a391d57ffc8d7154a534c0894d1057947ab/debian/patches/0003-disable-vpnkit.patch
https://salsa.debian.org/go-team/packages/rootlesskit/-/blob/74020a391d57ffc8d7154a534c0894d1057947ab/debian/patches/0004-Disable-lxcusernic.patch

These kinds of changes should be possible with go build tags like go build -tags "no_network_vpnkit no_network_lxcusernic"

Clarify that slirp4netns < v0.4.0 is no longer supported since RootlessKit v0.7.1

slirp4netns v0.3.X turned out not to work with RootlessKit because of lack of rootless-containers/slirp4netns@9115b34

k3s-io/k3s#1709 (comment)

This issue seems introduced in this commit (RootlessKit v0.7.1): 6ebc172

benchmark: more realistic workload

The current benchmark suite just measures throughput of ipef3 -c hostLoopbackIP.

rootlesskit/hack/test/docker-entrypoint.sh

Line 14 in 9a00d4a

$ROOTLESSKIT --net=slirp4netns $@ $IPERF3C 10.0.2.2

We should also measure apt-get-ish workloads with real repo servers on the Internet.

[VPNKit+kernel 4.19] CPU 100% on debian stretch

On debian stretch I see rootlesskit helper process going to 100% CPU quickly after launching rootless docker.

The process in question: /proc/self/exe --net=vpnkit --mtu=1500 --disable-host-loopback --copy-up=/etc --copy-up=/run /home/tonis/bin/dockerd-rootless.sh --experimental --iptables=false --storage-driver=vfs . Afaik we haven't reproduced it in other systems.

Linux dev3.localdomain 4.19.0-0.bpo.1-amd64 #1 SMP Debian 4.19.12-1~bpo9+1 (2018-12-30) x86_64 GNU/Linux

SIGQUIT: quit
PC=0x458d01 m=0 sigcode=0

goroutine 0 [idle]:
runtime.futex(0xb2f8e0, 0x80, 0x0, 0x0, 0x0, 0x4450f7, 0x0, 0x0, 0x7ffee9122d18, 0x40a4d2, ...)
	/usr/local/go/src/runtime/sys_linux_amd64.s:531 +0x21
runtime.futexsleep(0xb2f8e0, 0x0, 0xffffffffffffffff)
	/usr/local/go/src/runtime/os_linux.go:46 +0x4b
runtime.notesleep(0xb2f8e0)
	/usr/local/go/src/runtime/lock_futex.go:151 +0xa2
runtime.stopm()
	/usr/local/go/src/runtime/proc.go:2016 +0xe3
runtime.findrunnable(0xc00002a000, 0x0)
	/usr/local/go/src/runtime/proc.go:2487 +0x4dc
runtime.schedule()
	/usr/local/go/src/runtime/proc.go:2613 +0x13a
runtime.goexit0(0xc000078480)
	/usr/local/go/src/runtime/proc.go:2790 +0x1f2
runtime.mcall(0x0)
	/usr/local/go/src/runtime/asm_amd64.s:299 +0x5b

goroutine 1 [syscall]:
syscall.Syscall6(0xf7, 0x1, 0x47c8, 0xc000121580, 0x1000004, 0x0, 0x0, 0x60, 0xc00018c180, 0x0)
	/usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
os.(*Process).blockUntilWaitable(0xc0000265a0, 0x4, 0xc0001216e8, 0xc0000265a0)
	/usr/local/go/src/os/wait_waitid.go:31 +0x98
os.(*Process).wait(0xc0000265a0, 0xc00004e700, 0x5313ab, 0x0)
	/usr/local/go/src/os/exec_unix.go:22 +0x39
os.(*Process).Wait(0xc0000265a0, 0x839b68, 0x839b70, 0x839b60)
	/usr/local/go/src/os/exec.go:125 +0x2b
os/exec.(*Cmd).Wait(0xc00016e580, 0x0, 0x0)
	/usr/local/go/src/os/exec/exec.go:465 +0x5b
os/exec.(*Cmd).Run(0xc00016e580, 0x4, 0x4)
	/usr/local/go/src/os/exec/exec.go:309 +0x5c
github.com/rootless-containers/rootlesskit/pkg/child.Child(0x82a62c, 0x20, 0xc0000a0060, 0x4, 0x4, 0xc0000c6600, 0x0, 0x0)
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/pkg/child/child.go:221 +0x295
main.main.func2(0xc0000b6840, 0x0, 0x0)
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/cmd/rootlesskit/main.go:101 +0xd4
github.com/rootless-containers/rootlesskit/vendor/github.com/urfave/cli.HandleAction(0x79dfe0, 0xc0000a8300, 0xc0000b6840, 0x0, 0x0)
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/vendor/github.com/urfave/cli/app.go:490 +0xc8
github.com/rootless-containers/rootlesskit/vendor/github.com/urfave/cli.(*App).Run(0xc0000ae9c0, 0xc0000a0000, 0xa, 0xa, 0x0, 0x0)
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/vendor/github.com/urfave/cli/app.go:264 +0x59d
main.main()
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/cmd/rootlesskit/main.go:109 +0x93d

goroutine 4 [syscall]:
syscall.Syscall(0x0, 0x3, 0xc00017c000, 0x10000, 0xc0000c6040, 0x0, 0x48eb81)
	/usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5
syscall.read(0x3, 0xc00017c000, 0x10000, 0x10000, 0xc0000c6001, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:732 +0x5a
syscall.Read(0x3, 0xc00017c000, 0x10000, 0x10000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:172 +0x49
internal/poll.(*FD).Read(0xc00009eae0, 0xc00017c000, 0x10000, 0x10000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:165 +0x119
os.(*File).read(0xc00009c118, 0xc00017c000, 0x10000, 0x10000, 0x10000, 0xc00017c000, 0x7f37306175f0)
	/usr/local/go/src/os/file_unix.go:249 +0x4e
os.(*File).Read(0xc00009c118, 0xc00017c000, 0x10000, 0x10000, 0x7918a0, 0x1, 0xc00017c000)
	/usr/local/go/src/os/file.go:108 +0x69
github.com/rootless-containers/rootlesskit/vendor/github.com/jamescun/tuntap.(*device).Read(0xc0000a8e80, 0xc00017c000, 0x10000, 0x10000, 0x10000, 0x10000, 0x0)
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/vendor/github.com/jamescun/tuntap/device_linux.go:26 +0x4d
github.com/rootless-containers/rootlesskit/pkg/network/vpnkit.tap2vif(0xc0000b2be0, 0x7f373061b000, 0xc0000a8e80)
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/pkg/network/vpnkit/vpnkit.go:189 +0x95
created by github.com/rootless-containers/rootlesskit/pkg/network/vpnkit.startVPNKitRoutines
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/pkg/network/vpnkit/vpnkit.go:181 +0x537

goroutine 5 [IO wait]:
internal/poll.runtime_pollWait(0x7f37307c0118, 0x72, 0xc00004c3c8)
	/usr/local/go/src/runtime/netpoll.go:173 +0x66
internal/poll.(*pollDesc).wait(0xc000158098, 0x72, 0xffffffffffffff00, 0x8899a0, 0xaf5380)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9a
internal/poll.(*pollDesc).waitRead(0xc000158098, 0xc00019a000, 0x2, 0x8)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc000158080, 0xc00019a008, 0x2, 0x8, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x179
net.(*netFD).Read(0xc000158080, 0xc00019a008, 0x2, 0x8, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc00009c120, 0xc00019a008, 0x2, 0x8, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x68
io.ReadAtLeast(0x7f37307c0530, 0xc00009c120, 0xc00019a008, 0x2, 0x8, 0x2, 0x79be80, 0x1, 0xc00019a008)
	/usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(0x7f37307c0530, 0xc00009c120, 0xc00019a008, 0x2, 0x8, 0x0, 0x200, 0x0)
	/usr/local/go/src/io/io.go:329 +0x58
encoding/binary.Read(0x7f37307c0530, 0xc00009c120, 0x88e2a0, 0xb4c508, 0x787360, 0xc00019a000, 0x0, 0x0)
	/usr/local/go/src/encoding/binary/binary.go:171 +0x50a
github.com/rootless-containers/rootlesskit/vendor/github.com/moby/vpnkit/go/pkg/vmnet.(*Vif).Read(0xc0000b2be0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/vendor/github.com/moby/vpnkit/go/pkg/vmnet/vmnet.go:261 +0xa8
github.com/rootless-containers/rootlesskit/pkg/network/vpnkit.vif2tap(0x7f373061b020, 0xc0000a8e80, 0xc0000b2be0)
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/pkg/network/vpnkit/vpnkit.go:201 +0x2f
created by github.com/rootless-containers/rootlesskit/pkg/network/vpnkit.startVPNKitRoutines
	/tmp/tmp.OaOUjTeq2A/src/github.com/rootless-containers/rootlesskit/pkg/network/vpnkit/vpnkit.go:182 +0x597

rax    0xca
rbx    0xb2f7a0
rcx    0x458d03
rdx    0x0
rdi    0xb2f8e0
rsi    0x80
rbp    0x7ffee9122ce0
rsp    0x7ffee9122c98
r8     0x0
r9     0x0
r10    0x0
r11    0x286
r12    0x30
r13    0x11
r14    0x8833a4
r15    0x0
rip    0x458d01
rflags 0x286
cs     0x33
fs     0x0
gs     0x0

@AkihiroSuda

[VPNkit] rootless docker works only for one user (because of /tmp/go.pcap)

When working with https://get.dockerless.com/rootless I found that only one user could run the docker service at a time. The reason is that the first to write /tmp/go.pcap was the owner of that file, and because other users could not write it (because of they did not own it), they could not start docker.

I opened pull request #55 demonstrating a possible solution to this issue (which I implemented on my system to work around this issue).

support specifying mount propagation

Currently filesystems newly mounted on the host after RootlessKit has started cannot be seen from RootlessKit namespaces.

This could be solved by supporting non-private mount propagation.

Specify lxc config

I'm playing around with rootless docker and lxc, and It was working well. I wanted to attach the create LXC veth pair to my Open vSwitch. However it is difficult to work out which device has been connected to each "namespace".

I have not made any code changes to rootlesskit yet, but I was wondering if its just simply setting the following line to a real place

rootlesskit/pkg/network/lxcusernic/lxcusernic.go

Line 51 in 0359c83

dummyLXCPath := "/dev/null"

which as ~/.lxc/config and including the following config?

lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 00:16:3e:fe:d7:f0
lxc.network.name = veth0
lxc.network.veth.pair = veth-<a know name>

Or is there a more important reason this path is set to /dev/null?
Or is there a better way to control the host veth name?

Thanks for any input.

[VPNKit] panic: tap2vif: read: read /dev/net/tun: not pollable

panic: tap2vif: read: read /dev/net/tun: not pollable

goroutine 19 [running]:
github.com/rootless-containers/rootlesskit/pkg/network/vpnkit.tap2vif(0xc0001700a0, 0x7eff64137198, 0xc00000c200)
        /tmp/tmp.wZbjFCTJTg/src/github.com/rootless-containers/rootlesskit/pkg/network/vpnkit/vpnkit.go:195 +0x16f
created by github.com/rootless-containers/rootlesskit/pkg/network/vpnkit.startVPNKitRoutines
        /tmp/tmp.wZbjFCTJTg/src/github.com/rootless-containers/rootlesskit/pkg/network/vpnkit/vpnkit.go:185 +0x516

RootlessKit v0.9.2, dockerd-rootless.sh on moby/moby@37defbf .
Happened during early startup phase.

[docs] Running rootlesskit inside docker with few permissions

We want to run rootlesskit inside a docker container, to be able to run a docker daemon there (for integreation tests)

I know that I can run the container that is starting rootlesskit with seccomp=unconfined apparmor=unconfined systempaths=unconfined

This is removing a lot of docker security functionality. It should be documented on how to run the container with the least privileges required, especially which system calls it acutally needs / a custom seccomp profile.

benchmark: improve human-readability of CI log

The current CI log is hard to read

https://travis-ci.org/rootless-containers/rootlesskit/builds/640423841

remove dependency on /sbin/ip binary

`rootlesskit --debug` doesn't print slirp4netns stdout to debug log

rootlesskit/pkg/network/slirp4netns/slirp4netns.go

Lines 169 to 171 in 925dd4f

    
           // FIXME: Stdout doen't seem captured 
        
           cmd.Stdout = d.logWriter 
        
           cmd.Stderr = d.logWriter

stderr is printed as expected, but stdout is not.
slirp4netns: v1.0.1

failed to start the child: fork/exec /proc/self/exe: permission denied

I try using dockerd-rootless on a rancher cluster as for building docker images for CI/CD.
I use a Jenkins slave image as a base image (jenkins/jnlp-slave:3.27-1) where I installed docker rootless from https://get.docker.com/rootless.

When executing rootlesskit I have the following error :

$ rootlesskit sh
[rootlesskit:parent] error: failed to start the child: fork/exec /proc/self/exe: permission denied
$ rootlesskit -v
rootlesskit version 0.4.1

Static Binaries as part of release

I would be cool to have downloadable static binaries as part of release, rathe than compilling from souce.

Thanks

"child_pid" file should not be created until the child is fully configured

Currently, child_pid is created under the state directory before the child is fully created.

This causes race for nsenter -t $(cat child_pid) scripts.

rootlesskit-docker-proxy should report detailed error via FD 3

$ docker --context=rootless run -it -p 80:80 --rm alpine
docker: Error response from daemon: driver failed programming external connectivity on endpoint clever_euclid (77c992ce4056362ee89727f4d2d96c349bbb9013b14f4f2012412f2fbc94718e): Error starting userland proxy:.

rootlesskit-docker-proxy should report detailed error via FD 3.

NodePort management API

RootlessKit will provide implementation-agnostic REST API for exposing a port in the child netns to the parent netns, via an UNIX socket.

e.g. to expose 80/tcp as 8080/tcp in the host:

curl -X PUT -H Content-Type:application/json -d '{"childPort":80}' --unix-socket /run/user/1001/usernetes/rootlesskit/rootlesskit.sock http://v1/parentPorts/tcp/8080

Planned implementations:

`builtin`

Built-in implementation based on libnetwork proxy

`reexec-with-cgroups`

Re-exec builtin with cgroups

`vpnkit-native`

Uses VPNKit "VMN3T" API

`slirp4netns-native`

Uses slirp4netns builtin forwarder.
slirp4netns needs to be modified.

`exec-privileged-iptables`

Executes iptables with some SETUID/SETCAP helper.
Less secure but fast.

rootless-containers/usernetes#26

overlay2-fs driver not working properly on Debian 10

We are using Debian 10 Buster with kernel 4.19. As stated in the documentation this kernel allows to mount overlayfs in user namespace as non-root user.

Unfortunately when the docker daemon is started as rootless with overlay2 driver we are not able to pull any image from dockerhub and end up with the message described below. However the old overlay driver is working as expected.

Do you have any clue on this issue ? Has anyone encountered the same problem ?

Command to start docker as rootless with overlay2-fs

/home/testuser/bin/dockerd-rootless.sh --experimental --storage-driver=overlay2 --bip=100.64.1.0/16

Error when trying to pull an image

docker pull jboss/keycloak

Using default tag: latest
latest: Pulling from jboss/keycloak
600f7e2abab1: Pull complete 
6cd4d9d86398: Extracting  1.653kB/1.653kB
5402af5581b0: Download complete 
b91909199448: Download complete 
6ae51cb1a0de: Download complete 
failed to register layer: Error processing tar file(exit status 1): replaceDirWithOverlayOpaque("/root/buildinfo") failed: createDirWithOverlayOpaque("/root/rdwoo893274543") failed: failed to mkdir /root/rdwoo893274543/m/d: mkdir /root/rdwoo893274543/m/d: input/output error

When we use the old driver overlay, the pull is sucessfull, see below.

Command to start docker as rootless with overlay-fs

/home/testuser/bin/dockerd-rootless.sh --experimental --storage-driver=overlay --bip=100.64.1.0/16

Success when trying to pull an image

docker pull jboss/keycloak

Using default tag: latest
latest: Pulling from jboss/keycloak
600f7e2abab1: Pull complete 
6cd4d9d86398: Pull complete 
5402af5581b0: Pull complete 
b91909199448: Pull complete 
6ae51cb1a0de: Pull complete 
Digest: sha256:e9248bcf1d1444ab6d652e2302ef906321b9107b1a4d324aef2ab9479e6f2322
Status: Downloaded newer image for jboss/keycloak:latest
docker.io/jboss/keycloak:latest

As a side note here are the content of our sysctl.conf et modprobe.conf

/etc/sysctl.d/50-docker.conf

net.ipv4.ip_unprivileged_port_start=0
net.ipv4.ping_group_range = 0 2147483647
kernel.unprivileged_userns_clone=1

/etc/modprobe.d/overlay2.conf

options overlay permit_mounts_in_userns=1

mountSysfs needs to mount /sys/fs/selinux ?

rootlesskit/pkg/child/child.go

Lines 78 to 95 in cde143e

    
           // mountSysfs is needed for mounting /sys/class/net 
        
           // when netns is unshared. 
        
           func mountSysfs() error { 
        
           	tmp, err := ioutil.TempDir("/tmp", "rksys") 
        
           	if err != nil { 
        
           		return errors.Wrap(err, "creating a directory under /tmp") 
        
           	} 
        
           	defer os.RemoveAll(tmp) 
        
           	cmds := [][]string{ 
        
           		{"mount", "--rbind", "/sys/fs/cgroup", tmp}, 
        
           		{"mount", "-t", "sysfs", "none", "/sys"}, 
        
           		{"mount", "-n", "--move", tmp, "/sys/fs/cgroup"}, 
        
           	} 
        
           	if err := common.Execs(os.Stderr, os.Environ(), cmds); err != nil { 
        
           		return errors.Wrapf(err, "executing %v", cmds) 
        
           	} 
        
           	return nil 
        
           }

According to https://github.com/podenv/silverkube/blob/eb3fac03bdbcc7a2e9a25c255a8146287269a2df/silverkube.py#L132 , it seems the function needs to mount /sys/fs/selinux as well as /sys/fs/cgroup ?

cc @TristanCacqueray

systemd breaks our bind-mounted /etc/resolv.conf

When /etc/resolv.conf is a symlink to ../run/systemd/resolve/stub-resolv.conf, our bind-mounted /etc/resolv.conf (in our namespaces) is unexpectedly unmounted when /run/systemd/resolve/stub-resolv.conf is recreated.

Solution

Change

mount --bind /tmp/ourSlirpResolvConf /etc/resolv.conf

mount -t tmpfs none /run/systemd/resolve
echo dummy > /run/systemd/resolve/stub-resolv.conf
mount --bind /tmp/ourSlirpResolvConf /etc/resolv.conf

@cyphar PTAL? Is this correct way?

[rootlesskit:parent] error: failed to setup UID/GID map: failed to compute uid map: no matching entries in passwd file

Command producing the error was ./bin/dockerd-rootless.sh --experimental --storage-driver vfs

Trying to get this working on RHEL 7.6 kernel 3.10.0-957.1.3.el7.x86_64. We use NIS and SSSD so local /etc/passwd and /etc/group do not contain any network user specific information. ypcat passwd would be able to match uid.

Build fails on armv7

Since moby/moby#38050 our nightly builds for Docker CE have been failing with:

Install rootlesskit version 3c4582e950e3a67795c2832179c125b258b78124
+ echo 'Install rootlesskit version 3c4582e950e3a67795c2832179c125b258b78124'
+ git clone https://github.com/rootless-containers/rootlesskit.git /tmp/tmp.YNt7AorSyx/src/github.com/rootless-containers/rootlesskit
Cloning into '/tmp/tmp.YNt7AorSyx/src/github.com/rootless-containers/rootlesskit'...
+ cd /tmp/tmp.YNt7AorSyx/src/github.com/rootless-containers/rootlesskit
+ git checkout -q 3c4582e950e3a67795c2832179c125b258b78124
+ go build -ldflags= -o /build//rootlesskit github.com/rootless-containers/rootlesskit/cmd/rootlesskit
# github.com/rootless-containers/rootlesskit/pkg/network/iputils
pkg/network/iputils/iputils.go:18:12: constant 4294967295 overflows int

This error only occurs when we're running on armv7 (arm32) machines.

CC @AkihiroSuda

	// FIXME: Stdout doen't seem captured
	cmd.Stdout = d.logWriter
	cmd.Stderr = d.logWriter

	// mountSysfs is needed for mounting /sys/class/net
	// when netns is unshared.
	func mountSysfs() error {
	tmp, err := ioutil.TempDir("/tmp", "rksys")
	if err != nil {
	return errors.Wrap(err, "creating a directory under /tmp")
	}
	defer os.RemoveAll(tmp)
	cmds := [][]string{
	{"mount", "--rbind", "/sys/fs/cgroup", tmp},
	{"mount", "-t", "sysfs", "none", "/sys"},
	{"mount", "-n", "--move", tmp, "/sys/fs/cgroup"},
	}
	if err := common.Execs(os.Stderr, os.Environ(), cmds); err != nil {
	return errors.Wrapf(err, "executing %v", cmds)
	}
	return nil
	}

rootless-containers / rootlesskit Goto Github PK

rootlesskit's People

Contributors

Stargazers

Watchers

Forkers

rootlesskit's Issues

builtin

reexec-with-cgroups

vpnkit-native

slirp4netns-native

exec-privileged-iptables