jason-li-831202 / yolo-nas-onnxruntime Goto Github PK

This repo provides the C++ implementation of YOLO-NAS based on ONNXRuntime for performing object detection in real-time.Support float32/float16/int8 inference.

License: GNU General Public License v3.0

CMake 2.32% Shell 2.23% Python 23.86% C++ 71.59%

cpp onnxruntime python yolo-nas object-detection pytorch yolo

yolo-nas-onnxruntime's Introduction

YOLO-NAS-OnnxRuntime (FP32/FP16/INT8)

This repo provides the C++ implementation of YOLO-NAS based on ONNXRuntime for performing object detection in real-time.

➤ Contents

Dependecies
Build
Examples
Demo
References
License

➤ Dependecies

Python Install :

Quick installation from SuperGradients package
```
  pip install super_gradients==3.1.1
```
C++ Install :
- OpenCV 4.x
- ONNXRuntime 1.10+
- OS: Tested on Ubuntu 20.04 and Apple Silicon (M1)
- CUDA 11+ [Optional]

➤ Build

Rapidly build the project you can run the following commands:

# You can change ONNXRUNTIME_VERSION="x.x.x" in line 4.

./build.sh

If you want to use your own onnxruntime version, decompress the tgz library of onnxruntime :

onnxruntime repo: https://github.com/microsoft/onnxruntime/tags

tar zxvf onnxruntime-linux-x64-gpu-1.10.0.tgz

Next, don't forget to change ONNXRUNTIME_DIR cmake option:

mkdir build && cd build
cmake .. -D ONNXRUNTIME_DIR=path_to_onnxruntime -DCMAKE_BUILD_TYPE=Release
cmake --build .

➤ Examples

Comvert Pytorch to ONNX model :
```
# default coco model
python3 convertPytorchToONNX.py --input_model yolo_nas_s --output_dir ./models --img-size 640 640 

# your custom model
python3 convertPytorchToONNX.py --input_model yolo_nas_s --output_dir ./models --img-size 640 640 --checkpoint_path <path-to-pth-model> --class_names <path-to-names-file>
```
Description of all arguments:
- --input_model : Type contains { yolo_nas_s / yolo_nas_m / yolo_nas_l }
- --img-size : Set model input size (h, w)
- --output_dir : Directory for saving files, none means using the same path as the input model
- --half : Convert fp32 to fp16 model.
  
  [ Custom Model Args ]
- --checkpoint_path : The path with save the trained pth model
- --class_names : The path to class names file
  
  [ PTQ Args ]
- --int8 : Conver to int8.
- --calib_image_dir : if None will use dynamic quantization.
YOLO-NAS ~0.5 mAP by official information:

Model mAP Latency (ms)

YOLO-NAS S 47.5 3.21

YOLO-NAS M 51.55 5.85

YOLO-NAS L 52.22 7.87

YOLO-NAS S INT-8 47.03 2.36

YOLO-NAS M INT-8 51.0 3.78

YOLO-NAS L INT-8 52.1 4.78

Model	mAP	Latency (ms)
YOLO-NAS S	47.5	3.21
YOLO-NAS M	51.55	5.85
YOLO-NAS L	52.22	7.87
YOLO-NAS S INT-8	47.03	2.36
YOLO-NAS M INT-8	51.0	3.78
YOLO-NAS L INT-8	52.1	4.78

Run webcam source from CLI :

./demo --model_path ../models/yolo_nas_s.onnx --source 0 --class_names ../models/coco.names --gpu

Run video source from CLI :

./demo --model_path ../models/yolo_nas_s.onnx --source ../demo/video.mp4 --class_names ../models/coco.names --gpu

Run image source from CLI :

./demo --model_path ../models/yolo_nas_s.onnx --source ../demo/traffic.jpg --class_names ../models/coco.names --gpu

➤ Demo

➤ References

YOLO-Nas repo: https://github.com/Deci-AI/super-gradients

➤ License

WiFi Analyzer is licensed under the GNU General Public License v3.0 (GPLv3).

GPLv3 License key requirements :

Disclose Source
License and Copyright Notice
Same License
State Changes

yolo-nas-onnxruntime's People

Contributors

Stargazers

Watchers

Forkers

ocordeiro tienhoangvan pasquale90 ganbayard richiesui superrichiesui eamon-cai

yolo-nas-onnxruntime's Issues

doesnt work on custom dataset

Hello, this works perfect on default coco model but doesnt work on a custom one.
Did you try this yourself cause it does not work on custom model with checkpoint. Predictions don't work and are negative numbers.

When I try to run the code, I get always Ort::exception even running on image or video

Hi! I have an converted (not custom, default COCO) Yolo-NAS-S model from default .pth to ONNX. When I execute the program by using following arguments:

./demo -m ~/models/dnn/yolo_nas_s.onnx -s 0 -c ~/Downloads/coco.names

I get this error when trying to execute on video detection (this is whole log):

[INFO ] Onnxruntime Version:16
[INFO ] Inference device: CPU
[INFO ] Inference model: /home/uki/models/dnn/yolo_nas_s.onnx
[INFO ] ---------------- Input info --------------
[INFO ] Name [0]: input.1
[INFO ] Shape [0]: (1, 3, 640, 640, )
[INFO ] --------------- Output info --------------
[INFO ] Name [0]: 913
[INFO ] Shape [0]: (1, 8400, 4, )
[INFO ] Name [1]: 904
[INFO ] Shape [1]: (1, 8400, 80, )
[INFO ] Class num: -1
[INFO ] ==========================================
[INFO ] Model was initialized.
[INFO ] Current FPS : 30
terminate called after throwing an instance of 'Ort::Exception'
  what():  Invalid Output Name:h���
Aborted (core dumped)

And this error with image, even if given image is properly working, ends .jpg as code needs, and it is tested with OpenCV test code, what gives proper results. This is also an full log:

[INFO ] =============== Model info ===============
[INFO ] Onnxruntime Version:16
[INFO ] Inference device: CPU
[INFO ] Inference model: /home/uki/models/dnn/yolo_nas_s.onnx
[INFO ] ---------------- Input info --------------
[INFO ] Name [0]: input.1
[INFO ] Shape [0]: (1, 3, 640, 640, )
[INFO ] --------------- Output info --------------
[INFO ] Name [0]: 913
[INFO ] Shape [0]: (1, 8400, 4, )
[INFO ] Name [1]: 904
[INFO ] Shape [1]: (1, 8400, 80, )
[INFO ] Class num: -1
[INFO ] ==========================================
[INFO ] Model was initialized.
terminate called after throwing an instance of 'Ort::Exception'
  what():  Invalid Output Name:eg;*.jpg;*.jpe)

I am using the latest version of ONNX Runtime, 1.16.3.
The ONNX file is tested, it works properly.

So when I debugged with Visual Studio Code, I get the error specifically at this line, when it tries to get an outputNames.size(), e.g. when it runs an session.Run (get detection results):

std::vector<Ort::Value> outputTensors = this->session.Run(Ort::RunOptions{nullptr},
                                                              inputNames.data(),
                                                              inputTensors.data(),
                                                              inputNames.size(),
                                                              outputNames.data(),
                                                              outputNames.size() );

Do you know what maybe is issue? Thanks in advance. If you need more info, please leave a comment.

No instance of overloaded function "std::vector<_Ty, _Alloc>::push_back [with _Ty=Ort::Float16_t, _Alloc=std::allocator<Ort::Float16_t>]" matches the argument list

I am encountering the error in detector.cpp at this line

inputTensorValuesFp16.push_back(float32_to_float16(fp32));

I can see that the declaration of inputTensorValuesFp16 is
std::vector<Ort::Float16_t> inputTensorValuesFp16;

Also the function float32_to_float16 returns uint16_t
static uint16_t float32_to_float16(float& input_fp32)

Is this some bug? Or am i missing something here?

Help me with implementing this project.

It seems like a really good project. But since I am a beginner in C++, I am not sure how to implement this.

When I try the CLI, it shows ./demo is not a internal or recognized command.

Could you please provide me with a step by step procedure? I am familiar with Pytorch implementation of YOLO-NAS and would like to compare both these implementations.

Thanks and have a great day!

C# version

Hi,

is it possible to have a C# version for .Net user?

Thanks

Is compatible with latest onnxruntime (1.15.0) ?

Hello, I'm encountering an error while attempting to build with the latest version of onnxruntime on Apple Silicon. Unfortunately, previous versions of onnxruntime do not have a release specifically for Apple Silicon architecture.

Error:

[ 25%] Building CXX object CMakeFiles/demo.dir/src/main.cpp.o
[ 50%] Building CXX object CMakeFiles/demo.dir/src/detector.cpp.o
/Users/alan/Sites/YOLO-NAS-onnxruntime/src/detector.cpp:55:44: error: no member named 'GetInputName' in 'Ort::Session'
        inputNames.push_back(this->session.GetInputName(layer, allocator));
                             ~~~~~~~~~~~~~ ^
/Users/alan/Sites/YOLO-NAS-onnxruntime/src/detector.cpp:78:45: error: no member named 'GetOutputName' in 'Ort::Session'
        outputNames.push_back(this->session.GetOutputName(layer, allocator));
                              ~~~~~~~~~~~~~ ^
2 errors generated.
make[2]: *** [CMakeFiles/demo.dir/src/detector.cpp.o] Error 1
make[1]: *** [CMakeFiles/demo.dir/all] Error 2
make: *** [all] Error 2

About model chooses

There are only three choices in the Comvert Pytorch to ONNX model parameter, are there other models available, such as the basic yolov7 v8 model?