media-smart / volksdep Goto Github PK

when i run the sample code:

dummy_input = torch.ones(1, 3, 224, 224).cuda()
model = torchvision.models.resnet18().cuda().eval()
## build trt model with fp32 mode
trt_model = torch2trt(model, dummy_input)

raise the error

env:
cuda: 10.2
tensorrt: 8
pytorch: 1..0

I meet some errors when I try to run benchmark of volksdep like following. How to fix it?

there's a feature in TensorRT called ReformatFreeIO, is this supported?
and if supported, how to set dynamic range for tensors using this project?

TSN means Time-Sensitive Networking that is used in action recognition, this is the model i want to convert, does this tool support this convertion ? or it support other action recognition model ?

Volksdep installation fails with error "ERROR: No matching distribution found for tensorrt>=7.1.0.* ."
I have installed tensorrt on my system. still getting the error
I can import tensorrt in python code. Is there anything more i need to do ?

Hi! I want to convert my onnx to tensorrt model. The error is shown below. Could you please tell me how to solve it?

[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin ScatterND version 1
In node -1 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[TensorRT] ERROR: Network must have at least one output
[TensorRT] ERROR: Network validation failed.
Traceback (most recent call last):
File "zed_onnx2trt.py", line 9, in
trt_model = onnx2trt(model)
File "/home/nvidia/.local/lib/python3.6/site-packages/volksdep/converters/onnx2trt.py", line 119, in onnx2trt
trt_model = TRTModel(engine)
File "/home/nvidia/.local/lib/python3.6/site-packages/volksdep/converters/base.py", line 50, in init
self.context = self.engine.create_execution_context()
AttributeError: 'NoneType' object has no attribute 'create_execution_context'

Can the model only handle a single input?
I get an error when I change multiple inputs to a list.

Someone ever tried convert Bert model?

when I convert yolo4-tiny, this warning occurred,could you tell me how can I solve this problem? Thank you
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.

tks

I did follow the steps to install tensorRT 8.4.1 and pytorch. After execute the command pip install "git+https://github.com/Media-Smart/volksdep.git", it shows the error below:
Collecting git+https://github.com/Media-Smart/volksdep.git
Cloning https://github.com/Media-Smart/volksdep.git to /tmp/pip-req-build-fucad071
Running command git clone --filter=blob:none -q https://github.com/Media-Smart/volksdep.git /tmp/pip-req-build-fucad071
Resolved https://github.com/Media-Smart/volksdep.git to commit 9720829
Preparing metadata (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/tham/anaconda3/envs/volksdep/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-fucad071/setup.py'"'"'; file='"'"'/tmp/pip-req-build-fucad071/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vl0o25m0
cwd: /tmp/pip-req-build-fucad071/
Complete output (34 lines):
ERROR: Could not find a version that satisfies the requirement tensorrt>=7.1.0.* (from versions: 0.0.1.dev5, 0.0.1)
ERROR: No matching distribution found for tensorrt>=7.1.0.*
/home/tham/.local/lib/python3.6/site-packages/setuptools/installer.py:30: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
SetuptoolsDeprecationWarning,
Traceback (most recent call last):
File "/home/tham/.local/lib/python3.6/site-packages/setuptools/installer.py", line 82, in fetch_build_egg
subprocess.check_call(cmd)
File "/home/tham/anaconda3/envs/volksdep/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/tham/anaconda3/envs/volksdep/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpgvf7182z', '--quiet', 'tensorrt>=7.1.0.*']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-req-build-fucad071/setup.py", line 70, in
zip_safe=False,
File "/home/tham/.local/lib/python3.6/site-packages/setuptools/init.py", line 152, in setup
_install_setup_requires(attrs)
File "/home/tham/.local/lib/python3.6/site-packages/setuptools/init.py", line 147, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/home/tham/.local/lib/python3.6/site-packages/setuptools/dist.py", line 809, in fetch_build_eggs
replace_conflicting=True,
File "/home/tham/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 773, in resolve
replace_conflicting=replace_conflicting
File "/home/tham/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 1056, in best_match
return self.obtain(req, installer)
File "/home/tham/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 1068, in obtain
return installer(requirement)
File "/home/tham/.local/lib/python3.6/site-packages/setuptools/dist.py", line 877, in fetch_build_egg
return fetch_build_egg(self, req)
File "/home/tham/.local/lib/python3.6/site-packages/setuptools/installer.py", line 84, in fetch_build_egg
raise DistutilsError(str(e)) from e
distutils.errors.DistutilsError: Command '['/home/tham/anaconda3/envs/volksdep/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpgvf7182z', '--quiet', 'tensorrt>=7.1.0.*']' returned non-zero exit status 1.

WARNING: Discarding git+https://github.com/Media-Smart/volksdep.git. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
May I know how to solve this issue?
These are the command howI install TensorRT:
sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.2.0/local_installers/cuda-repo-ubuntu1804-11-2-local_11.2.0-460.27.04-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804-11-2-local_11.2.0-460.27.04-1_amd64.deb
sudo apt-key add /var/cuda-repo-ubuntu1804-11-2-local/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda
sudo apt-get install tensorrt-libs
sudo apt-get install tensorrt-dev
dpkg -l | grep TensorRT
ii libnvinfer-bin 8.4.1-1+cuda11.6 amd64 TensorRT binaries
ii libnvinfer-dev 8.4.1-1+cuda11.6 amd64 TensorRT development libraries and headers
ii libnvinfer-plugin-dev 8.4.1-1+cuda11.6 amd64 TensorRT plugin libraries
ii libnvinfer-plugin8 8.4.1-1+cuda11.6 amd64 TensorRT plugin libraries
ii libnvinfer-samples 8.4.1-1+cuda11.6 all TensorRT samples
ii libnvinfer8 8.4.1-1+cuda11.6 amd64 TensorRT runtime libraries
ii libnvonnxparsers-dev 8.4.1-1+cuda11.6 amd64 TensorRT ONNX libraries
ii libnvonnxparsers8 8.4.1-1+cuda11.6 amd64 TensorRT ONNX libraries
ii libnvparsers-dev 8.4.1-1+cuda11.6 amd64 TensorRT parsers libraries
ii libnvparsers8 8.4.1-1+cuda11.6 amd64 TensorRT parsers libraries
ii python3-libnvinfer 8.4.1-1+cuda11.6 amd64 Python 3 bindings for TensorRT
ii tensorrt 8.4.1.5-1+cuda11.6 amd64 Meta package for TensorRT
ii tensorrt-dev 8.4.1.5-1+cuda11.6 amd64 Meta package for TensorRT development libraries
ii tensorrt-libs 8.4.1.5-1+cuda11.6 amd64 Meta package for TensorRT runtime libraries

• pytorch version 1.2.0
• python 3.5.2(i changed the setup file to build this project)

try to convert deeplabv3+ model using this project ,got KeyError: 'upsample_bilinear2d';
follow this page to fix the onnx issue ,then got RuntimeError: ONNX export failed: Couldn't export operator aten::upsample_bilinear2d

any help?

Hi,
I'm trying to convert pytorch model (tinaface) to onnx format but it was fail.

Error log is as below:

root@1ce0ebf9a4bc:/workspace/vedadet# python tools/torch2onnx.py configs/trainval/tinaface/tinaface.py ../tinaface_r50_fpn_widerface.pth test.onnx --dummy_input_shape 3,640,640
Convert to Onnx with constant input shape 3,640,640 and opset version 9
Traceback (most recent call last):
File "tools/torch2onnx.py", line 65, in
main()
File "tools/torch2onnx.py", line 60, in main
verbose=args.verbose)
File "/opt/conda/lib/python3.6/site-packages/volksdep/converters/torch2onnx.py", line 62, in torch2onnx
dynamic_axes=dynamic_axes)
File "/opt/conda/lib/python3.6/site-packages/torch/onnx/init.py", line 172, in export
custom_opsets, enable_onnx_checker, use_external_data_format)
File "/opt/conda/lib/python3.6/site-packages/torch/onnx/utils.py", line 92, in export
use_external_data_format=use_external_data_format)
File "/opt/conda/lib/python3.6/site-packages/torch/onnx/utils.py", line 545, in _export
val_add_node_names, val_use_external_data_format, model_file_location)
RuntimeError: ONNX export failed: Couldn't export Python operator DeformConvFunction
Defined at:
/workspace/vedadet/vedadet/ops/dcn/deform_conv.py(302): forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(561): _slow_forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(575): call
/workspace/vedadet/vedadet/models/backbones/resnet.py(274): _inner_forward
/workspace/vedadet/vedadet/models/backbones/resnet.py(297): forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(561): _slow_forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(575): call
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/container.py(100): forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(561): _slow_forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(575): call
/workspace/vedadet/vedadet/models/backbones/resnet.py(630): forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(561): _slow_forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(575): call
/workspace/vedadet/vedadet/models/detectors/single_stage_detector.py(35): forward_impl
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(561): _slow_forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(575): call
/opt/conda/lib/python3.6/site-packages/torch/jit/init.py(394): wrapper
/opt/conda/lib/python3.6/site-packages/torch/jit/init.py(408): forward
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py(577): call
/opt/conda/lib/python3.6/site-packages/torch/jit/init.py(320): _get_trace_graph
/opt/conda/lib/python3.6/site-packages/torch/onnx/utils.py(319): _trace_and_get_graph_from_model
/opt/conda/lib/python3.6/site-packages/torch/onnx/utils.py(366): _model_to_graph
/opt/conda/lib/python3.6/site-packages/torch/onnx/utils.py(530): _export
/opt/conda/lib/python3.6/site-packages/torch/onnx/utils.py(92): export
/opt/conda/lib/python3.6/site-packages/torch/onnx/init.py(172): export
/opt/conda/lib/python3.6/site-packages/volksdep/converters/torch2onnx.py(62): torch2onnx
tools/torch2onnx.py(60): main
tools/torch2onnx.py(65):

This op isn't be support by volksdep, right?
Please help me confirm it.

Failed to convert my pytorch model to trt module using torch2trt. Maybe something wrong with the step that builder.build_engine(). My enviroment information is CUDA11.4, TensorRT8,2. Could you please help me with this problem?

File "/usr/anaconda3/envs/pytorch/lib/python3.8/site-packages/volksdep/converters/base.py", line 50, in init
self.context = self.engine.create_execution_context()
AttributeError: 'NoneType' object has no attribute 'create_execution_context'
[01/18/2022-16:53:01] [TRT] [E] [layers.h::setPostPadding::667] Error Code 3: API Usage Error (Parameter check failed at: /_src/build/cuda-11.4/8.2/x86_64/release/optimizer/api/layers.h::setPostPadding::667, condition: ((postPadding.nbDims == 2 || postPadding.nbDims == 3) && allDimsGtEq(postPadding, 0) && allDimsLtEq(postPadding, MAX_PADDING))
)
In node 53 (parseGraph): INVALID_NODE: Invalid Node - ConvTranspose_53
[layers.h::setPostPadding::667] Error Code 3: API Usage Error (Parameter check failed at: /_src/build/cuda-11.4/8.2/x86_64/release/optimizer/api/layers.h::setPostPadding::667, condition: ((postPadding.nbDims == 2 || postPadding.nbDims == 3) && allDimsGtEq(postPadding, 0) && allDimsLtEq(postPadding, MAX_PADDING))
)
[01/18/2022-16:53:01] [TRT] [E] 4: [network.cpp::validate::2633] Error Code 4: Internal Error (Network must have at least one output)

Process finished with exit code 1

我的环境：
python 3.6.9
pytorch 1.6.0
cuda 10.2
tensorrt 7.1.3.0

按照readme文档操作，我将pytorch模型直接转成trt模型，过程中也没有报错。
随后加载trt模型进行预测，图像加载到网络后，报错如下：

[TensorRT] ERROR: Parameter check failed at: engine.cpp::setBindingDimensions::1045, condition: profileMinDims.d[i] <= dimensions.d[i]

此时打印trt网络的输出并与pytorch网络的输出相比较，网络的输出也是不一致的。
我不知道应该怎么解决这个问题，希望您能帮我解答。