nagatoyuki0943 / anomalib-onnxruntime-cpp Goto Github PK

Loading multiple models into memory and performing inference to speed them up makes individual checks very long.

Is there a way to get the speed cut through other options?

尝试使用这个demo加载多个模型，总是初始化的时候报错，大概查了一下，大概是说Ort::Env是一个全局唯一的问题，请教一下，你有什么办法吗？下面是我找到相似的问题。
DefTruth/lite.ai.toolkit#8

请教一下你用的是onnxruntime-win-x64-gpu哪个版本？我onnxruntime-win-x64-gpu-1.14.1编译报错
TensorrtExecutionProvider CUDAExecutionProvider CPUExecutionProvider
2023-10-20 15:56:30.2506761 [W:onnxruntime:, session_state.cc:1136 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2023-10-20 15:56:30.2550638 [W:onnxruntime:, session_state.cc:1138 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
Number of inputs = 1
Number of output = 2
input_dims: -1 3 256 256
output_dims: -1 -1 -1 -1
output_dims: -1

Hello，I compile this project using onnxruntime 1.15.1 cpu and gpu version successfully, but both is error when run demo program.

你好，请问可以分享一下json文件吗，我想复现一下的项目

控制台信息：
CUDAExecutionProvider CPUExecutionProvider
Number of inputs = 1
Number of output = 2
input_dims: 1 3 224 224
output_dims: 1 1 224 224
output_dims: 1
2023-01-06 15:02:23.6623457 [E:onnxruntime:, sequential_executor.cc:369 onnxruntime::SequentialExecutor::Execute] Non-zero status code returned while running Pow node. Name:'Pow_74' Status Message: D:\a_work\1\s\onnxruntime\core\framework\bfc_arena.cc:342 onnxruntime::BFCArena::AllocateRawInternal Available memory of 0 is smaller than requested bytes of 1972518912

Non-zero status code returned while running Pow node. Name:'Pow_74' Status Message: D:\a_work\1\s\onnxruntime\core\framework\bfc_arena.cc:342 onnxruntime::BFCArena::AllocateRawInternal Available memory of 0 is smaller than requested bytes of 1972518912
这是cuda没调用起来么？用CPU可以测试，速度有点慢