agent fail to start with "Failed to open Intel SGX device" about enclave-cc HOT 11 CLOSED

agent still need these devices to be mounted

/dev/sgx dir volume mount alone is enough.

from enclave-cc.

check what is the minor for that device on your node from /proc/misc

from enclave-cc.

since we install the SGX SDK libs from their APT repository, we automatically get the latest version 2.18 which is broken due to intel/linux-sgx#920. Creating a symlink from the non-standard /dev/sgx/enclave -> /dev/sgx_enclave fixes the problem

from enclave-cc.

currently our e2e does not check with HW mode so this issue is not reflected in time in all the PR e2e test, this situation should be improved.

from enclave-cc.

The SDK fix is expected to be released maybe this week still.

from enclave-cc.

app container also has this issue:

root@iZ2ze49w79e4zvkn2mcbscZ:~# cat /var/log/pods/default_enclave-cc-pod_28c552b5-cc3f-4cf4-ab49-c958edd2416b/hello-world/0.log
2023-01-12T00:37:34.512107365+08:00 stderr F [get_driver_type /home/sgx/jenkins/ubuntuServer2004-release-build-trunk-218/build_target/PROD/label/Builder-UbuntuSrv20/label_exp/ubuntu64/linux-trunk-opensource/psw/urts/linux/edmm_utility.cpp:111] Failed to open Intel SGX device.
2023-01-12T00:37:34.512127777+08:00 stderr F [ERROR] occlum-pal: Failed to create enclave with error code 0x2006: Invalid SGX device. Please make sure SGX module is enabled in the BIOS, and install SGX driver afterwards. (line 152, file src/pal_enclave.c)

@mythi @haosanzi

from enclave-cc.

that means to workaround this bug, we need change both agent's config.json and app's yaml to add sgx devices:
for this workload POD definition hello_world-enc.yaml, the device addition example:

apiVersion: v1
kind: Pod
metadata:
  name: enclave-cc-pod
spec:
  containers:
  - image: docker.io/eqmcc/helloworld_enc
    name: hello-world
    workingDir: "/run/rune/boot_instance/"
    volumeMounts:
    - mountPath: /dev/sgx/enclave
      name: sgx-enclave
    - mountPath: /dev/sgx/provision
      name: sgx-provision
    securityContext:
      privileged: true
    resources:
      limits:
        sgx.intel.com/epc: 600Mi
    env:
    - name: OCCLUM_RELEASE_ENCLAVE
      value: "1"
    command:
    - /run/rune/boot_instance/build/bin/occlum-run
    - /bin/hello_world
  runtimeClassName: enclave-cc
  volumes:
  - name: sgx-enclave
    hostPath:
      path: /dev/sgx_enclave
  - name: sgx-provision
    hostPath:
      path: /dev/sgx_provision

from enclave-cc.

@hairongchen if you create these /dev/sgx/enclave -> /dev/sgx_enclave symlinks on the host and use v0.24 device plugin it will just work

from enclave-cc.

PR for temp fix: #90

from enclave-cc.

@hairongchen if you create these /dev/sgx/enclave -> /dev/sgx_enclave symlinks on the host and use v0.24 device plugin it will just work

Make sense. Since 0.24.0 intel sgx plugin mount /dev/sgx dir to /dev/sgx dir in container environment. Thank you very much.

from enclave-cc.

this would work for workload container, but agent still need these devices to be mounted in as it does not ask k8s to add sgx resource via plugin.

from enclave-cc.