Comments (8)
TPAT在生成Plugin的过程里,会多次使用shapeinference 推算出出对应算子的输入输出形状(shape-inference对无法预测的场景会使用onnx-runtime真实的跑一遍). 但CodeFormer.onnx这个文件似乎没有办法用Onnx-runtime跑起来,你可以先确保这个文件能用shape inference和onnx runtime跑起来吗?
shape infernece : http://www.xavierdupre.fr/app/onnxcustom/helpsphinx/api/onnx_python/shape_inference.html
Onnx-runtiem : https://onnxruntime.ai/docs/
from tpat.
TPAT在生成Plugin的过程里,会多次使用shapeinference 推算出出对应算子的输入输出形状(shape-inference对无法预测的场景会使用onnx-runtime真实的跑一遍). 但CodeFormer.onnx这个文件似乎没有办法用Onnx-runtime跑起来,你可以先确保这个文件能用shape inference和onnx runtime跑起来吗? shape infernece : http://www.xavierdupre.fr/app/onnxcustom/helpsphinx/api/onnx_python/shape_inference.html Onnx-runtiem : https://onnxruntime.ai/docs/
另外:对于比较大的onnx,我们比较建议可以手写一个onnx,包括你需要生成plugin的op,Shape保持一致,生成了对应的plugin之后,将这个比较大的onnx type改为plugin的Class name。这样onnx-parser也可以识别你这个plugin
from tpat.
from tpat.
TPAT在生成Plugin的过程里,会多次使用shapeinference 推算出出对应算子的输入输出形状(shape-inference对无法预测的场景会使用onnx-runtime真实的跑一遍). 但CodeFormer.onnx这个文件似乎没有办法用Onnx-runtime跑起来,你可以先确保这个文件能用shape inference和onnx runtime跑起来吗? shape infernece : http://www.xavierdupre.fr/app/onnxcustom/helpsphinx/api/onnx_python/shape_inference.html Onnx-runtiem : https://onnxruntime.ai/docs/
这是我的onnx测试代码,使用shape_Infer和onnxruntime均没有问题。这是一个超分辨率的模型
def onnx_infer():
# shape_infer
onnx_model = onnx.load("CodeFormer.onnx")
infer_shape = shape_inference.infer_shapes(onnx_model)
onnx.checker.check_model(onnx_model)
sess = onnxruntime.InferenceSession("CodeFormer.onnx")
image = cv2.imread("00_00.png")
image = cv2.resize(image, (512, 512), interpolation=cv2.INTER_LINEAR)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
image = image / 255
mean = np.array([0.5, 0.5, 0.5])
std = np.array([0.5, 0.5, 0.5])
image = (image - mean) / std
image = np.transpose(image, [2, 0, 1])
image = np.expand_dims(image, 0).astype(np.float32)
output = sess.run([], {"input": image})[0]
output = np.squeeze(output, 0)
output = np.clip(output, -1, 1)
output = ((output + 1) / 2) * 255
output = np.transpose(output, [1, 2, 0])
output = cv2.cvtColor(output, cv2.COLOR_RGB2BGR)
cv2.imwrite("test.png", output)
from tpat.
ScatterElement
我这里没有对应的一些环境,可以请你用shape_inference和onnxruntime跑一下half_model.onnx吗? half_model.onnx是CodeFormer.onnx 里从input截取到ScatterElements这个op的子图
from tpat.
加载了half_model后报一样的错误
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (Concat_52) Op (Concat) [ShapeInferenceError] Can't merge shape info. Both source and target dimension have values but they differ. Source=1 Target=2 Dimension=0
暂时我怀疑是动态batch的问题,但是我还没有定位到该问题是发生在模型的哪个环节。
我将代码内的dynamic_batch强制为false,并将onnx的导出设置为batch=1,成功生成了插件。
from tpat.
实际上TPAT的dynamic Batch的方案使用Padding的方式实现的。核心思路是对dynamic batch的onnx模型填充进batch维,生成了各自对应的plugin之后,用一个统一的plugin给拼起来。
对onnx model里的batch赋真实值的Code:python/onnx_to_plugin.py : add_explicit_bs 函数。
所以对于比较大的模型,例如整个图里bs所在的维度可能会发生改变,当用Shape-inference和Onnx-Runtime运行从input截取到目标Node的子图的时候,就可能会failed.
但是更简单的方式其实是 创建一个与CodeFormer.onnx有相同输入输出Shape的ScatterElements onnx算子,生成了plugin之后用于CodeFormer.onnx。
加载了half_model后报一样的错误
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (Concat_52) Op (Concat) [ShapeInferenceError] Can't merge shape info. Both source and target dimension have values but they differ. Source=1 Target=2 Dimension=0
暂时我怀疑是动态batch的问题,但是我还没有定位到该问题是发生在模型的哪个环节。 我将代码内的dynamic_batch强制为false,并将onnx的导出设置为batch=1,成功生成了插件。
from tpat.
实际上TPAT的dynamic Batch的方案使用Padding的方式实现的。核心思路是对dynamic batch的onnx模型填充进batch维,生成了各自对应的plugin之后,用一个统一的plugin给拼起来。 对onnx model里的batch赋真实值的Code:python/onnx_to_plugin.py : add_explicit_bs 函数。 所以对于比较大的模型,例如整个图里bs所在的维度可能会发生改变,当用Shape-inference和Onnx-Runtime运行从input截取到目标Node的子图的时候,就可能会failed. 但是更简单的方式其实是 创建一个与CodeFormer.onnx有相同输入输出Shape的ScatterElements onnx算子,生成了plugin之后用于CodeFormer.onnx。
加载了half_model后报一样的错误
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (Concat_52) Op (Concat) [ShapeInferenceError] Can't merge shape info. Both source and target dimension have values but they differ. Source=1 Target=2 Dimension=0
暂时我怀疑是动态batch的问题,但是我还没有定位到该问题是发生在模型的哪个环节。 我将代码内的dynamic_batch强制为false,并将onnx的导出设置为batch=1,成功生成了插件。
了解🫡
from tpat.
Related Issues (20)
- RandomNormal not supported for frontend ONNX HOT 1
- Support CUDA 11.5 and TensorRT 8.2.1.3? HOT 1
- 无法跑通 example HOT 1
- Support for dynamic shape ? HOT 2
- No radical Subgraph optimization for TensorRT HOT 1
- precision for one hot plugin is wrong HOT 2
- cuda kernel code generated by Ansor‘s search space will use shared memory optimization to auto tuning? HOT 1
- Is maintained of this repo? HOT 1
- Fail to run example test_onehot_dynamic_direct.py HOT 12
- Error when running one_hot example HOT 6
- support for one hot plugin with dynamic axis other than batch_size dim HOT 4
- Is there a talk or article about of the implementation of this project? HOT 1
- unsupported ptx version error
- out of memeory HOT 2
- so build succeed, tensorrt run error HOT 6
- Could you provide simple tutorial on how to run onnx_to_plugin for simple operator? HOT 4
- KeyError int8 HOT 1
- so build succeed, tensorrt run error one hot example HOT 11
- 好久不维护了吗?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tpat.