Comments (22)
deanofthewebb
commented on May 13, 2024
1
@philipp-schmidt Yes, it indeed works with grid
from yolov7.
deanofthewebb
commented on May 13, 2024
1
2022-07-10 08:57:41 - __main__ - INFO - Setting BuilderFlag.FP16
2022-07-10 08:57:41 - __main__ - DEBUG - === Network Description ===
2022-07-10 08:57:41 - __main__ - DEBUG - Input 0 | Name: images | Shape: (1, 3, 640, 640)
2022-07-10 08:57:41 - __main__ - DEBUG - Output 0 | Name: output | Shape: (1, 25200, 85)
2022-07-10 08:57:41 - __main__ - DEBUG - Output 1 | Name: onnx::Sigmoid_502 | Shape: (1, 3, 80, 80, 85)
2022-07-10 08:57:41 - __main__ - DEBUG - Output 2 | Name: onnx::Sigmoid_691 | Shape: (1, 3, 40, 40, 85)
2022-07-10 08:57:41 - __main__ - DEBUG - Output 3 | Name: onnx::Sigmoid_880 | Shape: (1, 3, 20, 20, 85)
from yolov7.
philipp-schmidt
commented on May 13, 2024
1
Thanks to @Linaom1214 in #66 I found the issue.
I was using nvcr.io/nvidia/pytorch:21.08-py3 as suggested in the README.
Using nvcr.io/nvidia/pytorch:22.04-py3 with pytorch 1.12.0a0+bd13bc6, the export works with --grid as well.
You have to run onnx-simplifier though, otherwise some of the resize operations were not working in TensorRT.
@WongKinYiu Is there a specific reason for using Pytorch 21.08 and not more recent?
@deanofthewebb thanks for your help!
from yolov7.
WongKinYiu
commented on May 13, 2024
1
We start this project at March 2021, and main code are finished at August 2021. Since baselines models are trained at the time, we have to compare our new methods in same environments.
from yolov7.
WongKinYiu
commented on May 13, 2024
.yaml could conv directly from specific layer, .cfg need route first then conv to the routed layer.
https://github.com/WongKinYiu/yolov7/blob/main/cfg/deploy/yolov7.yaml#L22
from yolov7.
deanofthewebb
commented on May 13, 2024
Ok thanks! @WongKinYiu
from yolov7.
philipp-schmidt
commented on May 13, 2024
@deanofthewebb
what are you using to convert to TensorRT? Where's that output from that you pasted?
from yolov7.
deanofthewebb
commented on May 13, 2024
@philipp-schmidt Deepstream
from yolov7.
philipp-schmidt
commented on May 13, 2024
I tried converting with trtexec (prebuilt in the TensorRT NGC image) and I'm unable to do so if the ONNX export is with --grid enabled.
Were you successful in converting to TensorRT with the Detect() layer enabled? Or are you using a TensorRT plugin?
Issue is described here: #66
from yolov7.
deanofthewebb
commented on May 13, 2024
@philipp-schmidt This one works: #57
from yolov7.
philipp-schmidt
commented on May 13, 2024
Thanks for the pointer.
It's not using the --grid option on ONNX export though if I see that correctly.
You tested it with --grid?
from yolov7.
philipp-schmidt
commented on May 13, 2024
Same issue unfortunately. Anything you changed to get it working on your end?
Export via: python models/export.py --weights yolov7.pt --grid
Conversion:
root@3aa30b614471:/workspace/yolov7# python deploy/TensoorRT/onnx_to_tensorrt.py --onnx yolov7.onnx --fp16 --explicit-batch -o yolov7.engine
Namespace(calibration_batch_size=128, calibration_cache='calibration.cache', calibration_data=None, debug=False, explicit_batch=True, explicit_precision=False, fp16=True, gpu_fallback=False, int8=False, max_batch_size=None, max_calibration_size=2048, onnx='yolov7.onnx', output='yolov7.engine', refittable=False, simple=False, strict_types=False, verbosity=None)
2022-07-10 07:53:52 - __main__ - INFO - TRT_LOGGER Verbosity: Severity.ERROR
2022-07-10 07:53:52 - __main__ - INFO - Setting BuilderFlag.FP16
[TensorRT] ERROR: [graphShapeAnalyzer.cpp::throwIfError::1306] Error Code 9: Internal Error (Mul_378: broadcast dimensions must be conformable
)
ERROR: Failed to parse the ONNX file: yolov7.onnx
In node 378 (parseGraph): INVALID_NODE: Invalid Node - Mul_378
[graphShapeAnalyzer.cpp::throwIfError::1306] Error Code 9: Internal Error (Mul_378: broadcast dimensions must be conformable)
from yolov7.
deanofthewebb
commented on May 13, 2024
Try it with batch size 1
…On Sun, Jul 10, 2022 at 12:57 AM philipp-schmidt ***@***.***> wrote:
Same issue unfortunately. Anything you changed to get it working on your
end?
Export via: python models/export.py --weights yolov7.pt --grid
Conversion:
***@***.***:/workspace/yolov7# python deploy/TensoorRT/onnx_to_tensorrt.py --onnx yolov7.onnx --fp16 --explicit-batch -o yolov7.engine
Namespace(calibration_batch_size=128, calibration_cache='calibration.cache', calibration_data=None, debug=False, explicit_batch=True, explicit_precision=False, fp16=True, gpu_fallback=False, int8=False, max_batch_size=None, max_calibration_size=2048, onnx='yolov7.onnx', output='yolov7.engine', refittable=False, simple=False, strict_types=False, verbosity=None)
2022-07-10 07:53:52 - __main__ - INFO - TRT_LOGGER Verbosity: Severity.ERROR
2022-07-10 07:53:52 - __main__ - INFO - Setting BuilderFlag.FP16
[TensorRT] ERROR: [graphShapeAnalyzer.cpp::throwIfError::1306] Error Code 9: Internal Error (Mul_378: broadcast dimensions must be conformable
)
ERROR: Failed to parse the ONNX file: yolov7.onnx
In node 378 (parseGraph): INVALID_NODE: Invalid Node - Mul_378
[graphShapeAnalyzer.cpp::throwIfError::1306] Error Code 9: Internal Error (Mul_378: broadcast dimensions must be conformable)
—
Reply to this email directly, view it on GitHub
<#41 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACPNYZM4LHZEVQ3IAVMR46DVTJ67ZANCNFSM527HCUOA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
from yolov7.
deanofthewebb
commented on May 13, 2024
python models/export.py --weights /content/yolov7.pt --grid
usr/local/lib/python3.7/dist-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.10) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
Namespace(batch_size=1, device='cpu', dynamic=False, grid=True, img_size=[640, 640], weights='/content/yolov7.pt')
YOLOR 🚀 v0.1-8-gdfc6817 torch 1.11.0+cu113 CPU
Fusing layers...
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
Model Summary: 306 layers, 36905341 parameters, 36905341 gradients
/usr/local/lib/python3.7/dist-packages/torch/functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2228.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
Starting TorchScript export with torch 1.11.0+cu113...
./models/yolo.py:49: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if self.grid[i].shape[2:4] != x[i].shape[2:4]:
TorchScript export success, saved as /content/yolov7.torchscript.pt
Starting ONNX export with onnx 1.12.0...
ONNX export success, saved as /content/yolov7.onnx
CoreML export failure: No module named 'coremltools'
Export complete (11.78s). Visualize with https://github.com/lutzroeder/netron.
python3 onnx_to_tensorrt.py --onnx yolov7.onnx -b 1 -vv --explicit-batch -o yolov7.onnx_b1_gpu0_fp16.engine --fp16 --explicit-precision
...
[07/08/2022-23:53:12] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +70, GPU +73, now: CPU 70, GPU 73 (MiB)
2022-07-08 23:53:12 - __main__ - INFO - Serializing engine to file: yolov7.onnx_b1_gpu0_fp16.engine
[07/08/2022-23:53:12] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[07/08/2022-23:53:12] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
from yolov7.
philipp-schmidt
commented on May 13, 2024
@deanofthewebb Super weird, doesn't work for me.
root@3aa30b614471:/workspace/yolov7# python models/export.py --weights yolov7.pt --grid
Namespace(batch_size=1, device='cpu', dynamic=False, grid=True, img_size=[640, 640], weights='yolov7.pt')
YOLOR 🚀 v0.1-11-ge342736 torch 1.10.0a0+3fd9dcf CPU
Fusing layers...
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
Model Summary: 306 layers, 36905341 parameters, 36905341 gradients, 104.5 GFLOPS
Starting TorchScript export with torch 1.10.0a0+3fd9dcf...
/workspace/yolov7/./models/yolo.py:49: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if self.grid[i].shape[2:4] != x[i].shape[2:4]:
TorchScript export success, saved as yolov7.torchscript.pt
Starting ONNX export with onnx 1.8.204...
ONNX export success, saved as yolov7.onnx
CoreML export failure: No module named 'coremltools'
Export complete (5.96s). Visualize with https://github.com/lutzroeder/netron.
root@3aa30b614471:/workspace/yolov7# python3 deploy/TensoorRT/onnx_to_tensorrt.py --onnx yolov7.onnx -b 1 --explicit-batch -o yolov7.onnx_b1_gpu0_fp16.engine --fp16 --explicit-precision
Namespace(calibration_batch_size=128, calibration_cache='calibration.cache', calibration_data=None, debug=False, explicit_batch=True, explicit_precision=True, fp16=True, gpu_fallback=False, int8=False, max_batch_size=1, max_calibration_size=2048, onnx='yolov7.onnx', output='yolov7.onnx_b1_gpu0_fp16.engine', refittable=False, simple=False, strict_types=False, verbosity=None)
2022-07-10 08:19:22 - __main__ - INFO - TRT_LOGGER Verbosity: Severity.ERROR
2022-07-10 08:19:23 - __main__ - INFO - Setting BuilderFlag.FP16
[TensorRT] ERROR: [graphShapeAnalyzer.cpp::throwIfError::1306] Error Code 9: Internal Error (Mul_378: broadcast dimensions must be conformable
)
ERROR: Failed to parse the ONNX file: yolov7.onnx
In node 378 (parseGraph): INVALID_NODE: Invalid Node - Mul_378
[graphShapeAnalyzer.cpp::throwIfError::1306] Error Code 9: Internal Error (Mul_378: broadcast dimensions must be conformable
)
from yolov7.
deanofthewebb
commented on May 13, 2024
Try simplifying the onnx model first:
https://www.convertmodel.com/#input=onnx&output=onnx
from yolov7.
philipp-schmidt
commented on May 13, 2024
I tried that as well :D
Let me double check I'm not missing anything - thanks for the help :)
from yolov7.
deanofthewebb
commented on May 13, 2024
@philipp-schmidt Double check your TensorRT version:
pip install nvidia-pyindex
pip install nvidia-tensorrt
Mine serialized in version 213:
Serialized Engine Version: 213
from yolov7.
philipp-schmidt
commented on May 13, 2024
I'm getting the exact same problem with trtexec of TensorRT 8.2.4
NVIDIA Release 22.04 (build 34850664)
NVIDIA TensorRT Version 8.2.4
from yolov7.
deanofthewebb
commented on May 13, 2024
@philipp-schmidt Try this
pip install nvidia-pyindex
pip install nvidia-tensorrt
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: nvidia-tensorrt in .../.pyenv/versions/3.8.2/lib/python3.8/site-packages **(8.4.1.5)**
Requirement already satisfied: nvidia-cublas-cu11 in ....pyenv/versions/3.8.2/lib/python3.8/site-packages (from nvidia-tensorrt) (2022.4.8)
Requirement already satisfied: nvidia-cuda-runtime-cu11 in .../.pyenv/versions/3.8.2/lib/python3.8/site-packages (from nvidia-tensorrt) (2022.4.25)
Requirement already satisfied: nvidia-cudnn-cu11 in .../.pyenv/versions/3.8.2/lib/python3.8/site-packages (from nvidia-tensorrt) (2022.5.19)
Requirement already satisfied: nvidia-cublas-cu117 in .../.pyenv/versions/3.8.2/lib/python3.8/site-packages (from nvidia-cublas-cu11->nvidia-tensorrt) (11.10.1.25)
Requirement already satisfied: nvidia-cuda-runtime-cu117 in .../.pyenv/versions/3.8.2/lib/python3.8/site-packages (from nvidia-cuda-runtime-cu11->nvidia-tensorrt) (11.7.60)
Requirement already satisfied: nvidia-cudnn-cu116 in .../.pyenv/versions/3.8.2/lib/python3.8/site-packages (from nvidia-cudnn-cu11->nvidia-tensorrt) **(8.4.0.27)**
Requirement already satisfied: wheel in .../.pyenv/versions/3.8.2/lib/python3.8/site-packages (from nvidia-cublas-cu117->nvidia-cublas-cu11->nvidia-tensorrt) (0.37.1)
Requirement already satisfied: setuptools in .../.pyenv/versions/3.8.2/lib/python3.8/site-packages (from nvidia-cublas-cu117->nvidia-cublas-cu11->nvidia-tensorrt) (41.2.0)
from yolov7.
philipp-schmidt
commented on May 13, 2024
If I create the TensorRT engine without --grid it works. My output layers look like this:
2022-07-10 08:40:37 - __main__ - DEBUG - === Network Description ===
2022-07-10 08:40:37 - __main__ - DEBUG - Input 0 | Name: images | Shape: (1, 3, 640, 640)
2022-07-10 08:40:37 - __main__ - DEBUG - Output 0 | Name: output | Shape: (1, 3, 80, 80, 85)
2022-07-10 08:40:37 - __main__ - DEBUG - Output 1 | Name: 516 | Shape: (1, 3, 40, 40, 85)
2022-07-10 08:40:37 - __main__ - DEBUG - Output 2 | Name: 530 | Shape: (1, 3, 20, 20, 85)
Can you post the respective output of your onnx_to_tensorrt for Network Description? If grid is working for you there should only be one output instead of 3.
from yolov7.
philipp-schmidt
commented on May 13, 2024
I ran onnx-simplify, but still the same issue.
Netron shows that in the simplified graph, the node in question is actually the multiplication with anchors.
Tensors [1,3,80,80,2] and [3,2] can not be multiplied and TRT can't broadcast it either, for a good reason.
I will investigate further. Still confused why I get different results.
from yolov7.
Related Issues (20)
- input normalization HOT 3
- V7x-seg HOT 1
- why the test result and valid result on the same dataset is different? HOT 1
- My device uses a Radeon Instinct MI25 MxGPU, is this compatible with Yolov7? HOT 1
- Need help with deployment HOT 7
- How to add more epochs and resume training (with optimizer state), after all training epochs are finished?
- Incurred problem during training on 4090
- CUDA out of memory during training HOT 2
- Yolov7 confusion matrix with background FP=1 and TN=0
- [bug] AttributeError: module 'numpy' has no attribute 'int' while training model on custom dataset for image segmentation task HOT 1
- Change from yolov3 to yolov7 HOT 2
- face recognition HOT 1
- shape inference of TRT::EfficientNMS_TRT type is missing HOT 1
- Custom dataset training vs Transfer-Learning HOT 16
- Transfer Learning for Custom Object Detection HOT 1
- Is incremental training possible?
- Hyper-parameter optimization of YOLOv7 model
- Real-time video detection detect.py
- Why yolov7-seg YOLOR head doesn't use implicitM ?
- How Confidence Score Calculated?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yolov7.