<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.github

cache反序列化问题已经修复。另外经排查，用户的测试代码中指针的用法会导致无法预测，需要进行修改。 <div class="highlight highlight

TensorRT cache file loading error on Windows about fastdeploy HOT 2 CLOSED

paddlepaddle commented on May 17, 2024

TensorRT cache file loading error on Windows

from fastdeploy.

Comments (2)

DefTruth commented on May 17, 2024

cache反序列化问题已经修复。另外经排查，用户的测试代码中指针的用法会导致无法预测，需要进行修改。

#include "fastdeploy/vision.h"

void TrtInfer(const std::string& model_file, const std::string& image_file) {
	struct fastdeploy::RuntimeOption roi_option;
	fastdeploy::vision::detection::YOLOv7End2EndTRT* model;

	roi_option.UseGpu();
	roi_option.UseTrtBackend();
	roi_option.SetTrtInputShape("images", { 1, 3, 640, 640 });
	roi_option.EnableTrtFP16();
	roi_option.SetTrtCacheFile("yolo_end2end.cache");
    
	// model =
	// 	&fastdeploy::vision::detection::YOLOv7End2EndTRT(model_file, "", roi_option);
        //  这行指针赋值代码用法修改成以下这句
	model = new fastdeploy::vision::detection::YOLOv7End2EndTRT(model_file, "", roi_option);
	if (!model->Initialized()) {
		std::cerr << "Failed to initialize." << std::endl;
		return;
	}
	auto im = cv::imread(image_file);
	fastdeploy::vision::DetectionResult res;
	if (!model->Predict(&im, &res)) {
		std::cerr << "Failed to predict." << std::endl;
		return;
	}
	std::cout << res.Str() << std::endl;
        delete model;
}

int main() {
      TrtInfer("yolov7-end2end-trt-nms.onnx", "000000014439.jpg");
      return 0;
}

首次运行会保存cache文件，log如下

[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(416)::fastdeploy::TrtBackend::BuildTrtEngine Start to building TensorRT Engine...
[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(488)::fastdeploy::TrtBackend::BuildTrtEngine TensorRT Engine is built succussfully.
[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(490)::fastdeploy::TrtBackend::BuildTrtEngine Serialize TensorRTEngine to local file yolo_end2end.cache.
[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(501)::fastdeploy::TrtBackend::BuildTrtEngine TensorRTEngine is serialized to local file yolo_end2end.cache, we can load this model from the seralized engine directly next time.
[INFO] fastdeploy/runtime.cc(289)::fastdeploy::Runtime::Init    Runtime initialized with Backend::TRT in Device::GPU.

第二次及之后运行，直接从cache中加载，加快初始化

[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(541)::fastdeploy::TrtBackend::CreateTrtEngineFromOnnx        Detect serialized TensorRT Engine file in yolo_end2end.cache, will load it directly.
[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(106)::fastdeploy::TrtBackend::LoadTrtCache   Build TensorRT Engine from cache file: yolo_end2end.cache with shape range information as below,
[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(109)::fastdeploy::TrtBackend::LoadTrtCache   Input name: images, shape=[1, 3, 640, 640], min=[1, 3, 640, 640], max=[1, 3, 640, 640]
[INFO] fastdeploy/runtime.cc(289)::fastdeploy::Runtime::Init    Runtime initialized with Backend::TRT in Device::GPU.

from fastdeploy.

WXL-steven commented on May 17, 2024

我遇到了相似的错误，问题似乎依旧存在？

此外，检查了trt_backend.cc文件，之前的pull #274 已被合并。

from fastdeploy.

TensorRT cache file loading error on Windows about fastdeploy HOT 2 CLOSED

Comments (2)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent