How to use containerd-shim-rune-v2? about inclavare-containers HOT 22 OPEN

@linux-gcc Hi, according to this comment

The design of Docker here also prevents being able to configure a shim... e.g. we send runcv1 types for the v1 shim, runcv2 types for the v2 shim... and those are the only types registered. We don't have a way to register other types and pass along config to the shim.

It is useless to change the containerd of dockerd, because although containerd affects what shim to use, in the docker scenario, dockerd is the real decision maker of what shim to use. Docker community has decided not to support shims other than runc v1/2, so the shim-rune adaptation to support docker will not be accepted by Docker community.

If the user wants to use other runtimes in docker, the user need to configure the config.json of
dockerd like（https://github.com/inclavare-containers/inclavare-containers#dockerd）

If you want to use shim-rune, please use crictl or kubectl to run the pod.

from inclavare-containers.

Thank you for your advice, the current latest version of k8s does not support docker. I will fix the mistake in the flowchart.

from inclavare-containers.

Hi, please follow this guide to install and configure the containerd, this guide provides how to install and use shim-rune, rune, occlum, etc to create a confidential computing Kubernetes cluster.

And then following this guide to run a Hello-world pod based on rune and shim-rune.

from inclavare-containers.

Is shim-rune necessary to use with k8s?
@haosanzi

from inclavare-containers.

No, for test purposes, you can use crictl instead of installing K8s environment to use shim-rune and rune.

from inclavare-containers.

Hi, we have provided a docker images to run shim-rune based on crictl environment, this image is for development propose.
After you start this images ,please run

cd /root/samples && ./clean.sh;
          crictl run --timeout 90s hello.yaml pod.yam

to start a helloworld pod.

from inclavare-containers.

Now, after I configured it according to the document you gave (just did not install k8s), but the container seems to be monitored by containerd-shim-runc-v2 instead of containerd-shim-rune-v2 after it is started. What is the reason for this? thanks !

from inclavare-containers.

Could you please provide your step to start the container?
You need to use crictl or kubectl to create a confidential computing pod based on shim-rune. Since you did not install k8s and crictl, how did you launch a pod?

from inclavare-containers.

1、I directly use the following command to start the container in the docker +containerd+rune+occum environment. At present, The container seems to be able to start successfully, but containerd pulls up shim-runc instead of shim-rune. Will shim-rune not be used in this scenario?
2、The specific commands are as follows: docker run -it --rm --runtime=rune -e ENCLAVE_TYPE=intelSgx -e ENCLAVE_RUNTIME_PATH=/opt/occlum/build/lib/libocclum-pal.so -e ENCLAVE_RUNTIME_ARGS=occlum_instance_test occlum-app

from inclavare-containers.

Hi, please use crictl or kubectl to create a confidential computing pod based on shim-rune. In the docker +containerd+rune+occum environment, please refer to guide to use rune.

The difference between these two scenarios, please refer to this document

from inclavare-containers.

1、I directly use the following command to start the container in the docker +containerd+rune+occum environment. At present, The container seems to be able to start successfully, but containerd pulls up shim-runc instead of shim-rune. Will shim-rune not be used in this scenario? 2、The specific commands are as follows: docker run -it --rm --runtime=rune -e ENCLAVE_TYPE=intelSgx -e ENCLAVE_RUNTIME_PATH=/opt/occlum/build/lib/libocclum-pal.so -e ENCLAVE_RUNTIME_ARGS=occlum_instance_test occlum-app

Hi @linux-gcc,

shim-rune、containerd and docker all are go-language projects. shim-rune is based on shim v2 API.

• the core files of shim-rune
• the core files of containerd-shim-runc-v2

However, the docker codes include containerd but don't include shim-rune. It means docker can't support the 3rd containerd-shim-v2 likes shim-rune

So

• you need to run shim-rune or containerd-shim-runc-v2 with k8s or crictl rather than docker.

Please follow the guides of @haosanzi, Thanks @linux-gcc .

from inclavare-containers.

It means that shim-rune will not be involved in the docker+containerd+rune+occlum scenario?

from inclavare-containers.

It means that shim-rune will not be involved in the docker+containerd+rune+occlum scenario?

Hi again @linux-gcc,

You are right, docker can't support the 3rd containerd-shim v2 API. So shim-rune(containerd-shim-rune-v2) will not be involved in the docker+containerd+rune+occlum scenario.

from inclavare-containers.

It means that shim-rune will not be involved in the docker+containerd+rune+occlum scenario?

Hi @linux-gcc ,

If you are a docker user, you don't care the shim-rune.
If you are a k8s or crictl user, shim-rune may play a key role in the whole k8s path. Please follow the guides of @haosanzi.

from inclavare-containers.

First of all thanks for your reply. I have the following two questions:

1. If the docker+containerd+rune+occlum scenario does not involve shim-rune, why is the configuration of shim-rune involved in this document（https://github.com/inclavare-containers/inclavare-containers#dockerd）?
   .................................................................................
   /etc/containerd/config.toml, on your system.

    [plugins.cri.containerd]
      ...
      [plugins.cri.containerd.runtimes.rune]
        runtime_type = "io.containerd.rune.v2"
   

then restart containerd on your system.
..................................................................................................
2. In the docker+containerd+rune+occlum scenario, after I use the above command to start the container, why do I see containerd-shim-runc-v2 monitoring the container? The relevant process information is as follows:
732357 ?Sl 0:00 /usr/local/bin/containerd-shim-runc-v2 -namespace moby -id 97a4e1abcace5d4a92071f9b837fdaecd34d28495d8f532c41564f2c619cdba8 -address /run/containerd/containerd.sock
732378 pts/0 Rsl+ 0:19 _ /usr/local/bin/rune init

from inclavare-containers.

First of all thanks for your reply. I have the following two questions:

1. If the docker+containerd+rune+occlum scenario does not involve shim-rune, why is the configuration of shim-rune involved in this document（https://github.com/inclavare-containers/inclavare-containers#dockerd）?
   .................................................................................
   /etc/containerd/config.toml, on your system.

    [plugins.cri.containerd]
      ...
      [plugins.cri.containerd.runtimes.rune]
        runtime_type = "io.containerd.rune.v2"
   

then restart containerd on your system. .................................................................................................. 2. In the docker+containerd+rune+occlum scenario, after I use the above command to start the container, why do I see containerd-shim-runc-v2 monitoring the container? The relevant process information is as follows: 732357 ?Sl 0:00 /usr/local/bin/containerd-shim-runc-v2 -namespace moby -id 97a4e1abcace5d4a92071f9b837fdaecd34d28495d8f532c41564f2c619cdba8 -address /run/containerd/containerd.sock 732378 pts/0 Rsl+ 0:19 _ /usr/local/bin/rune init

1. This is the configuration of containerd rather than dockerd, k8s also needs containerd, Thanks.

from inclavare-containers.

First of all thanks for your reply. I have the following two questions:

1. If the docker+containerd+rune+occlum scenario does not involve shim-rune, why is the configuration of shim-rune involved in this document（https://github.com/inclavare-containers/inclavare-containers#dockerd）?
   .................................................................................
   /etc/containerd/config.toml, on your system.

    [plugins.cri.containerd]
      ...
      [plugins.cri.containerd.runtimes.rune]
        runtime_type = "io.containerd.rune.v2"
   

then restart containerd on your system. .................................................................................................. 2. In the docker+containerd+rune+occlum scenario, after I use the above command to start the container, why do I see containerd-shim-runc-v2 monitoring the container? The relevant process information is as follows: 732357 ?Sl 0:00 /usr/local/bin/containerd-shim-runc-v2 -namespace moby -id 97a4e1abcace5d4a92071f9b837fdaecd34d28495d8f532c41564f2c619cdba8 -address /run/containerd/containerd.sock 732378 pts/0 Rsl+ 0:19 _ /usr/local/bin/rune init

Hi again @linux-gcc,

2. Thanks for your information, In the newer docker, docker updated the vendor containerd version to v1.3.x and now the vendor containerd version is v1.6.1. So the newer docker can support containerd-shim-runc-v2 as you said.

`io.containerd.runc.v2` is available since containerd v1.3.0.

shim-rune is a 3rd shim runtime, we need to adapt to the docker and pull request like the issue moby/moby#42244 in docker.

from inclavare-containers.

First of all thanks for your reply. I have the following two questions:

1. If the docker+containerd+rune+occlum scenario does not involve shim-rune, why is the configuration of shim-rune involved in this document（https://github.com/inclavare-containers/inclavare-containers#dockerd）?
   .................................................................................
   /etc/containerd/config.toml, on your system.

    [plugins.cri.containerd]
      ...
      [plugins.cri.containerd.runtimes.rune]
        runtime_type = "io.containerd.rune.v2"
   

then restart containerd on your system. .................................................................................................. 2. In the docker+containerd+rune+occlum scenario, after I use the above command to start the container, why do I see containerd-shim-runc-v2 monitoring the container? The relevant process information is as follows: 732357 ?Sl 0:00 /usr/local/bin/containerd-shim-runc-v2 -namespace moby -id 97a4e1abcace5d4a92071f9b837fdaecd34d28495d8f532c41564f2c619cdba8 -address /run/containerd/containerd.sock 732378 pts/0 Rsl+ 0:19 _ /usr/local/bin/rune init

Hi again @linux-gcc,

2. Thanks for your information, In the newer docker, docker updated the vendor containerd version to v1.3.x and now the vendor containerd version is v1.6.1. So the newer docker can support containerd-shim-runc-v2 as you said.

`io.containerd.runc.v2` is available since containerd v1.3.0.

shim-rune is a 3rd shim runtime, we need to adapt to the docker and pull request like the issue moby/moby#42244 in docker.

It means that shim-rune does not support the docker scene yet.? Is the community considering shim-rune adaptation to support docker?

from inclavare-containers.

@linux-gcc Hi, according to this comment

The design of Docker here also prevents being able to configure a shim... e.g. we send runcv1 types for the v1 shim, runcv2 types for the v2 shim... and those are the only types registered. We don't have a way to register other types and pass along config to the shim.

It is useless to change the containerd of dockerd, because although containerd affects what shim to use, in the docker scenario, dockerd is the real decision maker of what shim to use. Docker community has decided not to support shims other than runc v1/2, so the shim-rune adaptation to support docker will not be accepted by Docker community.

If the user wants to use other runtimes in docker, the user need to configure the config.json of dockerd like（https://github.com/inclavare-containers/inclavare-containers#dockerd）

If you want to use shim-rune, please use crictl or kubectl to run the pod.

Hi again @linux-gcc ,

@haosanzi is the maintainer of shim-rune, please refer to the guide above. Thanks @linux-gcc

from inclavare-containers.

thanks!@hustliyilin @haosanzi I roughly understand what you mean.

by the way ,What is the specific function of shim-rune?

from inclavare-containers.

containerd-shim-rune-v2 is a shim for rune.

In addition to implementing Shim Runtime v2 API, containerd-shim-rune also undertakes advanced functions of Enclave management, such as Bundle conversion, Enclave signature, remote attestation and other functions.

Please refer to this document for information about shim-rune

from inclavare-containers.

If docker does not support shim-rune, shouldn't dockerd be involved in the following flowchart?
https://github.com/inclavare-containers/inclavare-containers/blob/master/shim/docs/images/shim-rune.png

from inclavare-containers.