Installing ONNXRUNTIME on Jetson Xavier Orin (CUDA 12.2) about jetson-containers HOT 5 OPEN

@huzaifahshamim this container includes TensorRT: https://github.com/dusty-nv/jetson-containers/tree/master/packages/onnxruntime

Or you can grab the wheels from http://jetson.webredirect.org/
Or if building onnxruntime from jetson-containers, it will rebuild the wheels for a custom configuration when necessary.

from jetson-containers.

Thank you for your quick response @dusty-nv .

Attempting to access jetson.webredirect.org gives me this error:

Hmmm… can't reach this page jetson.webredirect.org’s server IP address could not be found.
Try:

Search the web for jetson webredirect org
Checking the connection
Checking the proxy, firewall, and DNS settings

I had some additional dependencies I needed installed, so I did not want to solely use that container.

I also had a few follow ups:

1. My cuda version on the Orin is 12.2 which means that onnxruntime 17.0 should work correct? However, I still run into the cublass 11.0 errors. So, if I was to use the container, which one would I even use?
2. If I dont want to use the container, how can I access the .org website you linked?
3. What is the difference between the builder and nonbuilder tags?

Update: I believe I answered 1: Basically using $(autotag onnxruntime) let me know which container image is compatible.

Does this image also contain the devel version for tensorrt as that is what I also need. I would like to be able to use trtexec in the docker container that I spin up

from jetson-containers.

@huzaifahshamim yes I use the tensorrt-devel libraries and tensorrt-python bindings in jetson-containers, and jetson-containers will also build onnxruntime for you if needed.

from jetson-containers.

Sounds great, thank you! All is working well so far. I am running into an import error and was wondering what the best method to solve this would be.

I am attempting to replicate the inferencing done here: https://github.com/NVIDIA/TensorRT/blob/main/samples/python/efficientnet/infer.py

On line 24, from cuda import cudart is called, and I get a ModuleNotFound error on that line when I attempt to run infer.py in my container. I know that pip3 install cuda is not the right way to do it. What is the best way to do so?

from jetson-containers.

@huzaifahshamim for that you would want to include the cuda-python package or wheels in your container:

• https://github.com/dusty-nv/jetson-containers/tree/master/packages/cuda/cuda-python
• http://jetson.webredirect.org/jp6/cu122/cuda-python/12.2.0+0.g2ae98f9.dirty

from jetson-containers.