Error in nvidia-device-plugin pod. about k8s-device-plugin HOT 2 OPEN

@premalathak12 is the nvidia runtime configured as the default runtime in crio? If not, a runtime class must be created and associated with the nvidia runtime, and the runtime class specified when reploying the plugin.

from k8s-device-plugin.

@elezar Thanks for the quick reply.
Output for runtime command in both master and worker-gpu node :
crictl config --get runtime-endpoint
unix:///var/run/crio/crio.sock

root@ami:/home/ami# grep runtime /etc/crio/crio.conf
[crio.runtime]
[crio.runtime.runtimes]
[crio.runtime.runtimes.nvidia]
runtime_path = "/usr/bin/nvidia-container-runtime"
runtime_type = "oci"

I ran these two commands :
_Configuring CRI-O
Configure the container runtime by using the nvidia-ctk command:

sudo nvidia-ctk runtime configure --runtime=crio
The nvidia-ctk command modifies the /etc/crio/crio.conf file on the host. The file is updated so that CRI-O can use the NVIDIA Container Runtime.

Restart the CRI-O daemon:

sudo systemctl restart crio_

This section has no data :
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/sample-workload.html#running-sample-workloads-with-containerd-or-cri-o
can you suggest any sample workload for this to verify runtime ?

from k8s-device-plugin.