Docker 20.10.5 on armhf platforms about wdpksrc HOT 33 OPEN

@gabrielitos87,

I got as far as you did with diagnosing it with strace as well. I also tried "gdb", but not an expert in it and not sure it works well for troubleshooting issues with "golang" compiled applications.

Cheers,

JediNite

from wdpksrc.

Maybe this discussion and especially the solution at the end might be the way to go: moby/moby#40733
It suggests to use the raspbian sources instead of debian ...

from wdpksrc.

I managed to have working binaries!

After quite a number of tests, I finally found the winning recipe:

1. I used my Raspberry Pi 3 with Rasbpian Buster to compile armhf natively
2. I cloned the https://github.com/moby/moby repository
3. Checked out the 20.10 branch

root@raspy:~/moby# git branch -a
* 20.10
  master
  remotes/origin/1.12.x
  remotes/origin/1.13.x
  remotes/origin/17.03.x
  remotes/origin/17.04.x
  remotes/origin/17.05.x
  remotes/origin/19.03
  remotes/origin/20.10
  remotes/origin/HEAD -> origin/master
  remotes/origin/docs
  remotes/origin/master
  remotes/origin/revert-39415-master

4. Thinking about @JediNite comment, I figured that the broken official builds started approximately to be released when Debian Buster was stable. So I modified the Dockerfile to use Debian Stretch as base image, plus the following changes that allow to use it with the selected Moby branch.

diff --git a/Dockerfile b/Dockerfile
index f5ec77836b..4ac9fb7ebd 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -8,7 +8,7 @@ ARG DEBIAN_FRONTEND=noninteractive
 ARG VPNKIT_VERSION=0.5.0
 ARG DOCKER_BUILDTAGS="apparmor seccomp"
 
-ARG BASE_DEBIAN_DISTRO="buster"
+ARG BASE_DEBIAN_DISTRO="stretch"
 ARG GOLANG_IMAGE="golang:${GO_VERSION}-${BASE_DEBIAN_DISTRO}"
 
 FROM ${GOLANG_IMAGE} AS base
@@ -23,6 +23,7 @@ ARG DEBIAN_FRONTEND
 # Install dependency packages specific to criu
 RUN --mount=type=cache,sharing=locked,id=moby-criu-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-criu-aptcache,target=/var/cache/apt \
+        rm -rf /var/lib/apt/lists/ && \
         apt-get update && apt-get install -y --no-install-recommends \
             libcap-dev \
             libnet-dev \
@@ -85,6 +86,7 @@ FROM debian:${BASE_DEBIAN_DISTRO} AS frozen-images
 ARG DEBIAN_FRONTEND
 RUN --mount=type=cache,sharing=locked,id=moby-frozen-images-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-frozen-images-aptcache,target=/var/cache/apt \
+       rm -rf /var/lib/apt/lists/ && \
        apt-get update && apt-get install -y --no-install-recommends \
            ca-certificates \
            curl \
@@ -110,6 +112,7 @@ RUN dpkg --add-architecture armel
 RUN dpkg --add-architecture armhf
 RUN --mount=type=cache,sharing=locked,id=moby-cross-true-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-cross-true-aptcache,target=/var/cache/apt \
+        rm -rf /var/lib/apt/lists/ && \
         apt-get update && apt-get install -y --no-install-recommends \
             crossbuild-essential-arm64 \
             crossbuild-essential-armel \
@@ -122,6 +125,7 @@ ARG DEBIAN_FRONTEND
 RUN echo 'deb http://deb.debian.org/debian buster-backports main' > /etc/apt/sources.list.d/backports.list
 RUN --mount=type=cache,sharing=locked,id=moby-cross-false-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-cross-false-aptcache,target=/var/cache/apt \
+        rm -rf /var/lib/apt/lists/ && \
         apt-get update && apt-get install -y --no-install-recommends \
             binutils-mingw-w64 \
             g++-mingw-w64-x86-64 \
@@ -141,6 +145,7 @@ ARG DEBIAN_FRONTEND
 RUN echo 'deb http://deb.debian.org/debian buster-backports main' > /etc/apt/sources.list.d/backports.list
 RUN --mount=type=cache,sharing=locked,id=moby-cross-true-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-cross-true-aptcache,target=/var/cache/apt \
+        rm -rf /var/lib/apt/lists/ && \
         apt-get update && apt-get install -y --no-install-recommends \
             libapparmor-dev:arm64 \
             libapparmor-dev:armel \
@@ -167,8 +172,10 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
 
 FROM dev-base AS containerd
 ARG DEBIAN_FRONTEND
+RUN echo 'deb http://deb.debian.org/debian stretch-backports main' > /etc/apt/sources.list.d/backports.list
 RUN --mount=type=cache,sharing=locked,id=moby-containerd-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-containerd-aptcache,target=/var/cache/apt \
+        rm -rf /var/lib/apt/lists/ && \
         apt-get update && apt-get install -y --no-install-recommends \
             libbtrfs-dev
 ARG CONTAINERD_COMMIT
@@ -226,6 +233,7 @@ ARG DEBIAN_FRONTEND
 ARG TINI_COMMIT
 RUN --mount=type=cache,sharing=locked,id=moby-tini-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-tini-aptcache,target=/var/cache/apt \
+        rm -rf /var/lib/apt/lists/ && \
         apt-get update && apt-get install -y --no-install-recommends \
             cmake \
             vim-common
@@ -268,6 +276,7 @@ RUN ldconfig
 # Do you really need to add another package here? Can it be done in a different build stage?
 RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-dev-aptcache,target=/var/cache/apt \
+        rm -rf /var/lib/apt/lists/ && \
         apt-get update && apt-get install -y --no-install-recommends \
             apparmor \
             aufs-tools \
@@ -330,6 +339,7 @@ ENTRYPOINT ["hack/dind"]
 FROM dev-systemd-false AS dev-systemd-true
 RUN --mount=type=cache,sharing=locked,id=moby-dev-aptlib,target=/var/lib/apt \
     --mount=type=cache,sharing=locked,id=moby-dev-aptcache,target=/var/cache/apt \
+        rm -rf /var/lib/apt/lists/ && \
         apt-get update && apt-get install -y --no-install-recommends \
             dbus \
             dbus-user-session \

5. make build followed by make binary , waiting for the binaries to be created.

6. Move the binaries to the WD NAS and tadaa:

root@MyCloudEX2Ultra test # ./dockerd -v
Docker version dev, build 88bd96d6e5

The files that I generated are quite large and are available from my Dropbox. Can we package them in the WD format, @stefaang ?

Just for completeness, this is what I also tried but didn't work:

• Compile the moby/moby binaries with the original "buster" setting, straight from git clone or selecting the branch 20.10.
• Compile moby/moby forcing GOARCH=arm and GOARM=6 or GOARM=5 (so theoretically for armhfv6 armhfv5).
• Compiling manually without docker, following e.g. https://gist.github.com/cwgem/c913c80dcb8eeef38abc30ff3abf1750

from wdpksrc.

@gabrielitos87,

I've updated the old Github repo I had with details based on your findings and a new build procedure and script along with a copy of the binaries. Check it out at https://github.com/JediNite/docker-ce-WDEX4100-binaries.

Cheers,

JediNite

from wdpksrc.

@JediNite ,

It works! I checked out the commit, I created the package with build.sh and I installed in my EX2 Ultra. The flag --add-host host.docker.internal:host-gateway is accepted, so the version is confirmed!

Thanks!

from wdpksrc.

I got 20.10.7 and extract by
tar -xzvf /shares/hd2/docker/docker-20.10.7.tgz -C /mnt/HD/HD_a2/Nas_Prog/docker
docker -v still gives 20.10.7
but now unable to start transmission.
OCI runtime create failed. no space left on device

update. restarted and updated portainer and transmission works ok. thanks.

from wdpksrc.

I found why the mismatch happened, because of the if condition in the install.sh script:

if [ ${ARCH} != "x86_64" ]; then
	# Update the "ARCH" to "armhf" so it matches the docker download site
	# Versions above "19.03.8" do not have a working "dockerd" binary on WD EX4100
	ARCH="armhf"
	VERSION="19.03.8"
fi

Do you know why later versions are not working? How can I help?

from wdpksrc.

from wdpksrc.

Thanks for your answer.

I made some investigations too. I can confirm that downloading docker-20.10.5.tgz manually in the My Cloud EX2 Ultra and running ./dockerd -v also provides Segmentation Fault.

However, I tried something more: I did the same on my Raspberry Pi 3 (which is also armhf) and I get the same Segmentation Fault. Moreover, I tried also docker-20.10.2.tgz , and that also gives Segmentation Fault.

I did the last test because Docker 20.10.2 is the one available from the official repositories (through the script get-docker.sh): that works fine! You can appreciate the difference:

root@raspy:/home/gabrielitos/test/docker# dockerd -v
Docker version 20.10.2, build 8891c58
root@raspy:/home/gabrielitos/test/docker# ./dockerd -v
Segmentation fault

Following some Google digging, I found that this debugging can help, but this is above my current level of understanding.

root@raspy:/home/gabrielitos/test/docker# strace -f ./dockerd -v
execve("./dockerd", ["./dockerd", "-v"], 0xffc17a88 /* 17 vars */) = 0
syscall_0x101cc(0x37b0d6, 0x158031, 0x1, 0x18ef8f0, 0x101c4, 0x18c461d) = -1 ENOSYS (Function not implemented)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x4f00000} ---
+++ killed by SIGSEGV +++
Segmentation fault

Any guess?

from wdpksrc.

I created a bug report in the Docker repository: docker/for-linux#1226
@JediNite, can you maybe confirm happening also on your platform?

from wdpksrc.

@stefaang

Thanks for your answer. I checked the post too, but it seems that was a repository mistake: the Debian instead of the Raspbian package for was used (and those are packages with dynamically linked libraries, so we cannot use them here, right?).

I couldn't find how the Raspbian package was compiled: since it is hosted in the official Docker repository, I guess they used the standard source code...

from wdpksrc.

@stefaang

I tried to compile on my (poor old) Raspberry Pi 3 the whole Docker following the standard procedure (https://oyvindsk.com/writing/docker-build-from-source). After some hours, I obtained some binaries which are still providing Illegal Instruction!

I couldn't find how the Raspberry compilation should be treated differently than the normal compilation, and it is not working using the correct procedure from scratch. So I opened an issue on the moby tracker:
moby/moby#42212

from wdpksrc.

https://github.com/tttapa/RPi-Cpp-Toolchain
Did you try compiling docker with this toolchain? It claims to build working packages for ARMv6.
FYI, I don't think the EX2 ultra has hardfloat support enabled in its kernel.

from wdpksrc.

I am currently trying to build the static binaries of dockerd and the rest on my Rpi3 using a manual procedure similar to this:
https://gist.github.com/cwgem/c913c80dcb8eeef38abc30ff3abf1750

The main difference here is that I am also compiling the go compiler: I hope this makes a difference.

As I wrote on the Moby tracker, I compiled the docker using the automatic procedure (make binaries) from their git, but it resulted again in a dockerd providing "Illegal instructions". I managed also to cross compile forcing an armv5 and armv6 versions, but still the same problem.

from wdpksrc.

@gabrielitos87

Good work in tracking that down. I copied to binaries over to my EX4100 from your dropbox and tried to run "dockerd -v" as well and it works. I used to build binaries for the EX4100 on OS3 because "seccomp" was not enabled in the kernel, so can probably go back and have a go with the changes you have provided as well.

Have you updated the issue that you created on moby/moby as well ?

Cheers,

Jedinite

from wdpksrc.

@JediNite

Yes, I updated the issue in moby/moby but there was no reaction.. Maybe that's not depending on them, but on the golang buster Docker image.

How can the docker binaries be included in the WD package? Shall i just put them in the folder before calling build.sh?

from wdpksrc.

@gabrielitos87,

If you go back and have a look at https://github.com/WDCommunity/wdpksrc/blob/f6ef810c78a75fbb94541e5df4395a104ecf8655/wdpk/docker/install.sh I used to host the bundled binaries I used to build for EX4100 on another Github Repo. You could probably do something similar.

Cheers,

JediNite

from wdpksrc.

Hi all,

Commit 9726307 should now address this issue.

Cheers,

JediNite

from wdpksrc.

The flag --add-host host.docker.internal:host-gateway is accepted, so the version is confirmed!

It should also be possible to run "docker version", "docker --version" and "dockerd -v" and all of these should show version 20.10.5 as well.

from wdpksrc.

@JediNite

Yes, that shows the correct version too! But I thought that number was hardcoded by this:

make VERSION=${GEN_STATIC_VER} binary

What is the procedure now for this issue? Should a pull request be made?

from wdpksrc.

Great work @gabrielitos87 !
Don't worry about the binary size. It might be possible though to wrap the compile process in a single docker / make file, so the CI can build it.
I'm working on the website revamp the coming weeks in my very spare time.. it should be live by end of the month.

from wdpksrc.

What is the procedure now for this issue? Should a pull request be made?

@gabrielitos87

Check out the "quick and dirty" build script I made in https://github.com/JediNite/docker-ce-WDEX4100-binaries/blob/master/build.sh. One of the variables it takes in is the target build version to get from the docker-cli and moby/moby GitHub projects.

More then happy for someone to take this and try and automate it more.

Cheers,

JediNite

from wdpksrc.

Great work @gabrielitos87 !
Don't worry about the binary size. It might be possible though to wrap the compile process in a single docker / make file, so the CI can build it.
I'm working on the website revamp the coming weeks in my very spare time.. it should be live by end of the month.

i have pr2100
do you make new version docker and release?
now docker is so old version
so i want update but i don' know what i do

i hope release new version

from wdpksrc.

current docker has libseccomp problem with transmission.
https://forum.openmediavault.org/index.php?thread/38387-tranmission-using-stacks-unable-to-download/
new release fix this issue?

from wdpksrc.

@htogether,

Thanks to the work from @gabrielitos87, we were able to get docker working on the armhf platforms and part of getting this to work was to change the Ubuntu release used to build docker from stretch to buster. I saw as per https://docs.linuxserver.io/faq#libseccomp that the fixes involve installing a version of libseccomp2 which appears to have been sourced from buster. We would need to check if a similar package is available in stretch, update the "Dockerfile.patch" (which can be located at https://github.com/JediNite/docker-ce-WDEX4100-binaries) to include libseccomp2 in the "apt-get-install" statements and do some testing to see if this then allows firstly a successful docker compile and secondly a working transmission container.

Anyone else got thoughts on a different approach ?

Cheers,

JediNite

from wdpksrc.

@htogether

I've just put some binaries for 20.10.6 on my binaries repo (https://github.com/JediNite/docker-ce-WDEX4100-binaries/releases/tag/v20.10.6). Do you want to try these and see if the same issue is in 20.10.6 ? I did get an error in the patching process for the Dockerfile as some of the lines have changed between 20.10.5 and 20.10.6 and need to check these out further.

Cheers,

JediNite

from wdpksrc.

I have mirror gen2.

from wdpksrc.

@htogether,

The packages I release should work for any WD armhf platform, so if your gen2 is also arm based, it is worth a shot.

Reading through this a bit more, the fixes suggested seem to involve installing "libseccomp2" within the HOST OS itself. WD do not really allow access to the firmware files in order to add the package. They do occasionally release the source code packages and provide tools to build your own firmware file, but can be a bit hit and miss and has potential to brick your NAS if done incorrectly.

A "workaround" for your issue if it is seccomp related, might be to see if you can start the container with seccomp disabled.

Cheers,

JediNite

from wdpksrc.

any tips to installation? I am not familiar with os5. I used to use omv

from wdpksrc.

used ssh to extract and copy but write error.
root@mcmg2 tmp # tar xzvf docker-20.10.6.tgz
docker/
docker/docker
tar: write error: No space left on device

from wdpksrc.

Hi,

You can't extract this into /tmp as it is by design a small filesystem. This has to be extracted to on of the data drives. If you have installed a previous version of the docker packages, you can update the contents in /mnt/HD/HD_a2/Nas_Prog/docker/docker for example.

JediNite

from wdpksrc.

I’m trying to build a newer Docker (25.0.4 is current), but Stretch is no so old that it’s barely supported (apt repos archived, GoLang doesn’t support it anymore).

Is there a way to build with a more modern version of Debian?

from wdpksrc.

@koehn, scratch what I said before. I just tried using the standard docker binaries available from https://download.docker.com/linux/static/stable/armhf/ on my EX4100 and they worked fine, so there was no need to compile them. I am not sure if this would be the same for the EX2Ultra though, but maybe worth trying.

root@WDMyCloudEX4100 docker # docker version
Client:
Version: 26.1.2
API version: 1.45
Go version: go1.21.10
Git commit: 211e74b
Built: Wed May 8 13:59:49 2024
OS/Arch: linux/arm
Context: default

Server: Docker Engine - Community
Engine:
Version: 26.1.2
API version: 1.45 (minimum version 1.24)
Go version: go1.21.10
Git commit: ef1912d
Built: Wed May 8 13:28:49 2024
OS/Arch: linux/arm
Experimental: false
containerd:
Version: v1.7.15
GitCommit: 926c9586fe4a6236699318391cd44976a98e31f1
runc:
Version: 1.1.12
GitCommit: v1.1.12-0-g51d5e94
docker-init:
Version: 0.19.0
GitCommit: de40ad0

from wdpksrc.