compiled under cuda-11, tensorrt 8.2

$ ./test/test_model
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from PointPillars
[ RUN      ] PointPillars.__build_model__
../model/cbgs_pp_multihead_backbone.trt
 |￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣>
 | ../model/cbgs_pp_multihead_pfe.trt >
 |＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿>
             (\__/) ||
             (•ㅅ•) ||
             / 　 づ
WARNING: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.1 but loaded cuBLAS/cuBLAS LT 11.2.0
WARNING: TensorRT was linked against cuDNN 8.2.1 but loaded cuDNN 8.0.3
 |￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣>
 | ../model/cbgs_pp_multihead_backbone.trt >
 |＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿>
             (\__/) ||
             (•ㅅ•) ||
             / 　 づ
WARNING: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.1 but loaded cuBLAS/cuBLAS LT 11.2.0
WARNING: TensorRT was linked against cuDNN 8.2.1 but loaded cuDNN 8.0.3
WARNING: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.1 but loaded cuBLAS/cuBLAS LT 11.2.0
WARNING: TensorRT was linked against cuDNN 8.2.1 but loaded cuDNN 8.0.3
WARNING: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.1 but loaded cuBLAS/cuBLAS LT 11.2.0
WARNING: TensorRT was linked against cuDNN 8.2.1 but loaded cuDNN 8.0.3
ERROR: 1: [genericReformat.cu::executeMemcpy::1334] Error Code 1: Cuda Runtime (invalid argument)
GPUassert: an illegal memory access was encountered /home/work/pointpillars_multihead_40fps/pointpillars/pointpillars.cc 420

Hi, there is an error occurred when I do cmake
Error is as shown below.

============================================================================================
Building tests

-- BUILD SUMMARY
-- CMAKE_GENERATOR : Unix Makefiles
-- Compiler ID : GNU
-- Build type : Release
-- Build shared libs : ON
-- Use double for kernel: OFF
-- Build tests : ON

-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
NVINFER
linked by target "pointpillars" in directory /home/civeh/workspace/PointPillars_MultiHead_40FPS/pointpillars
linked by target "test_model" in directory /home/civeh/workspace/PointPillars_MultiHead_40FPS/test
NVONNXPARSERS
linked by target "pointpillars" in directory /home/civeh/workspace/PointPillars_MultiHead_40FPS/pointpillars
linked by target "test_model" in directory /home/civeh/workspace/PointPillars_MultiHead_40FPS/test

-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.

Please help

the lidar coordinate system of nuscenes is different from that of kitti.
If my point cloud coordinate system changes, does the pre-processing code need to be modified ?

I also used openPCdet for training, because the number of categories changed, I modified some parameters of post-processing, but the output of the model was all wrong boxes.
May I ask what needs to be modified in the pre-processed code?

Hi,
shall you update the onnx file address?
the old address was invalid....

我的模型是用kitti训练的，我将kitti配置文件改成muiltihead了，点云维度是4维，改了前处理和后处理部分参数，将num_gather_feature = 11改成10，kNumPointFeature = 5改成了4，类别是两种类别， dev_pfe_gather_feature_把第五行删掉了，最终预测结果全是错的或者为空。是点云维度原因吗？

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from PointPillars
[ RUN ] PointPillars.build_model
../model/robo_rpn/kitti_rpn_multihead_backbone.trt
|￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣>
| ../model/robo_rpn/kitti_rpn_multihead_pfe.trt >
|＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿>
(_/) ||
(•ㅅ•) ||
/ 　 づ
WARNING: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.1 but loaded cuBLAS/cuBLAS LT 11.2.1
|￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣>
| ../model/robo_rpn/kitti_rpn_multihead_backbone.trt >
|＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿>
(_/) ||
(•ㅅ•) ||
/ 　 づ
WARNING: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.1 but loaded cuBLAS/cuBLAS LT 11.2.1
WARNING: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.1 but loaded cuBLAS/cuBLAS LT 11.2.1
WARNING: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.1 but loaded cuBLAS/cuBLAS LT 11.2.1
4
true1true------------------------------------
Module Time

Preprocess 0.328729 ms
Pfe 1.92755 ms
Scatter 0.27791 ms
Backbone 3.28076 ms
Postprocess 1.63506 ms
Summary 7.45741 ms

4
true1true------------------------------------
Module Time

Preprocess 0.326377 ms
Pfe 1.91881 ms
Scatter 0.03204 ms
Backbone 3.17447 ms
Postprocess 1.62952 ms
Summary 7.08602 ms

4
true1true------------------------------------
Module Time

Preprocess 0.343313 ms
Pfe 1.92314 ms
Scatter 0.91915 ms
Backbone 3.18731 ms
Postprocess 1.60187 ms
Summary 7.98153 ms

4
true1true------------------------------------
Module Time

Preprocess 0.328076 ms
Pfe 1.91939 ms
Scatter 0.032131 ms
Backbone 3.17698 ms
Postprocess 1.65143 ms
Summary 7.11405 ms

4
true1true------------------------------------
Module Time

Preprocess 0.321695 ms
Pfe 1.91993 ms
Scatter 0.274737 ms
Backbone 3.17974 ms
Postprocess 1.6171 ms
Summary 7.31876 ms

4
true1true------------------------------------
Module Time

Preprocess 0.31875 ms
Pfe 1.9192 ms
Scatter 0.031389 ms
Backbone 3.1751 ms
Postprocess 1.58972 ms
Summary 7.03859 ms

4
true1true------------------------------------
Module Time

Preprocess 0.325528 ms
Pfe 1.92111 ms
Scatter 0.262098 ms
Backbone 3.17563 ms
Postprocess 1.6107 ms
Summary 7.30026 ms

4
true1true------------------------------------
Module Time

Preprocess 0.329693 ms
Pfe 1.91792 ms
Scatter 0.030285 ms
Backbone 3.17318 ms
Postprocess 1.67297 ms
Summary 7.1286 ms

4
true1true------------------------------------
Module Time

Preprocess 0.319577 ms
Pfe 1.92314 ms
Scatter 0.276512 ms
Backbone 3.17424 ms
Postprocess 1.60376 ms
Summary 7.30224 ms

4
true1true------------------------------------
Module Time

Preprocess 0.321262 ms
Pfe 1.91856 ms
Scatter 0.031172 ms
Backbone 3.16954 ms
Postprocess 1.62139 ms
Summary 7.06617 ms

[ OK ] PointPillars.build_model (1574 ms)
[----------] 1 test from PointPillars (1574 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (1574 ms total)
[ PASSED ] 1 test.

I used the provided onnx and run it in xavier in MAXN mode.
Running onnx model, takes about 130ms.
Running trt model, takes about 130ms. I'm sure it had used GPU.

这个速度
正常吗？

Module Time

Preprocess 0.44171 ms
Pfe 12.0893 ms
Scatter 0.118499 ms
Backbone 2.84478 ms
Postprocess 39.299 ms
Summary 54.8017 ms

num_objects----------------------------
2000
[ OK ] PointPillars.build_model (850 ms)
[----------] 1 test from PointPillars (850 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (850 ms total)
[ PASSED ] 1 test.

您好，请问在原版的训练代码中修改了哪里才可以转化为tensorrt，用最新的openPCDet v0.5.0训练的模型（cuda11.3,pytorch1.10,spconv2.1)可以用您这套c++代码部署吗？

Hello, there was an error while converting onnx to trt on agx orin ：ERROR] 2: [utils.cpp::checkMemLimit::380] Error Code 2: Internal Error (Assertion upperBound != 0 failed. Unknown embedded device detected. Please update the table with the entry: {{2055, 14, 32}, 24429},)
terminate called after throwing an instance of 'std::runtime_error'
what(): Failed to create object
Aborted (core dumped)
May I ask what error this is?

Our own dataset has been trained on openpcdet and I got the engine files. But I have trouble to understand the structure of this work. I will appreciate you if you could write a tutorial about how to run this work on our own dataset and then we could continue our research and cite your work on pointpillars.
Thanks for your useful work!

Link to the nuscenes_10sweeps_points.txt data is broken. Link leads to google drive and gives an error "File is in owner's trash".
Can the file be made downloadable again, so it's possible to follow steps in readme?

Can you tell me after write all the information down in bootstrap.yaml, how to do the inference.

Hi I have the same configuration as mentioned but I am getting the following error which running the test suite.

ERROR: ../rtSafe/coreReadArchive.cpp (38) - Serialization Error in verifyHeader: 0 (Version tag does not match)
ERROR: INVALID_STATE: std::exception
ERROR: INVALID_CONFIG: Deserialize the cuda engine failed.
ERROR: failed to build engine parser
Segmentation fault (core dumped)

在pth转onnx的过程中，执行https://github.com/hova88/OpenPCDet里的python trans_backbone_multihead.py
报错Exporting the operator atan2 to ONNX opset version 10 is not supported.
请问有没有遇到类似的问题？该怎么处理？

Great work! have you tested this model on Jetson devices like TX2 or Xavier ? or Do you have any plan to implement it in the future ?

成功了！谢谢分享

Hi, first of all, thank you for sharing your code.

I wonder that in pointpillars.cc, line 167~ shows initiate anchor sizes from yaml file. But i cannot find using that vectors in this repository.

Could you explain about this part?

Thank you.

I also encounter some problems when try to utilize this project on my own datasets.The format of my datasets is the way Openpcdet needs and I have transfered it to kitti format.It is trained well on 3 categories(car,pedestrain,cyclist) just with little changes on "cfgs/kitti_models/pointpillar.yaml".But when I try to apply my own datasets with cbgs_pp_multihead.yaml then something goes wrong.I think the problem is the different between nuscences and kitti and the change of the number of categories. Can you give me some hint of how to change cbgs_pp_multihead.yaml to train my datasets（3 categories）.Thanks!

runtime log:
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from PointPillars
[ RUN ] PointPillars.build_model
unknown file: Failure
C++ exception with description "bad file: ../bootstrap.yaml" thrown in the test body.
[ FAILED ] PointPillars.build_model (0 ms)
[----------] 1 test from PointPillars (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] PointPillars.build_model

1 FAILED TEST

terminate called after throwing an instance of 'pwgen::PwgenException'
what(): Driver error:
Aborted (core dumped)

Hello, we are Meituan Vision Team and are currently hiring. We are very interested to have you on board. Please contact zhangbo97 AT meituan DOT com.

在cmake时候报错

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
NVINFER

Hello everyone. In the hova88/openpcdet, i can convert multihead-model to onnx. Can I convert a model of singlehead witch is trained with kitti format?

Thank you for your great source code..
When I run the example more than one time, I received very random results. Can you explain the reason and give me some ideas to fix this bug?

你好，为什么postprocess.cu中的cls_pred_0等指针变量中的值不能访问，拷贝到host也不能访问

您好，我在执行 cmake .. 时，产生以下错误

Building tests
-- 

-- BUILD SUMMARY
--   CMAKE_GENERATOR      : Unix Makefiles
--   Compiler ID          : GNU
--   Build type           : Release
--   Build shared libs    : ON
--   Use double for kernel: OFF

--   Build tests          : ON
-- 

-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
NVINFER
    linked by target "pointpillars" in directory /home/cavata/project/OpenPCDet/PointPillars_MultiHead_40FPS/pointpillars
    linked by target "test_model" in directory /home/cavata/project/OpenPCDet/PointPillars_MultiHead_40FPS/test

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

请问您是否有相应的解决方案，十分感谢！

RPN_HEAD_CFGS: [
{
'HEAD_CLS_NAME': [ 'vehicle' ],
},
{
'HEAD_CLS_NAME': [ 'car' ],
},
{
'HEAD_CLS_NAME': [ 'pedestrian'],
},
{
'HEAD_CLS_NAME': [ 'cyclist' ],
},
]

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from PointPillars
[ RUN ] PointPillars.build_model
./models/custom_data_multihead_backbone.trt
unknown file: Failure
C++ exception with description "bad file: ./pointpillars/cfgs/pointpillar_sanyi_multi.yaml" thrown in the test body.
[ FAILED ] PointPillars.build_model (1 ms)
[----------] 1 test from PointPillars (1 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (1 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] PointPillars.build_model

1 FAILED TEST

Hi, thanks for your sharing.
Which GPU did you use for testing the performance?

我在融入跟踪算法的时候发现多了很多类别，想确认一下。

RPN_HEAD_CFGS: [
{
'HEAD_CLS_NAME': [ 'vehicle' ],
},
{
'HEAD_CLS_NAME': [ 'car' ],
},
{
'HEAD_CLS_NAME': [ 'pedestrian'],
},
{
'HEAD_CLS_NAME': [ 'cyclist' ],
},
]

hi, thanks for nice work.
I got some error when I run onnx2trt cbgs_pp_multihead_backbone.onnx -o cbgs_pp_multihead_backbone.trt -b 1 -d 16

(base) ➜  default git:(main) onnx2trt cbgs_pp_multihead_backbone.onnx -o cbgs_pp_multihead_backbone.trt -b 1 -d 16
----------------------------------------------------------------
Input filename:   cbgs_pp_multihead_backbone.onnx
ONNX IR version:  0.0.6
Opset version:    10
Producer name:    pytorch
Producer version: 1.7
Domain:
Model version:    0
Doc string:
----------------------------------------------------------------
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to create object
[1]    19112 abort (core dumped)  onnx2trt cbgs_pp_multihead_backbone.onnx -o cbgs_pp_multihead_backbone.trt -b 1 -d

my onnx2trt version

(base) ➜  default git:(main) onnx2trt -V                                                                     
Parser built against:
  ONNX IR version:  0.0.6
  TensorRT version: 7.1.3

and I found some issue about this. But I'm not found solution. any suggestion about this?
Crash when model with cast #406
run onnx2trt ERROR FAILED_ALLOCATION: std::bad_alloc #549

Module Time

Preprocess 0.440277 ms
Pfe 13.8402 ms
Scatter 0.11535 ms
Backbone 348.759 ms
Postprocess 40.2281 ms
Summary 403.392 ms

num_objects----------------------------

Hello,
Thank you for the great work you've done!

I'm a beginner, and I currently have a model that I trained by myself based on the KITTI dataset using the methods provided by the OpenPCDet project team. However, it doesn't work properly with your code.

I tried to train a multi-head model using the KITTI dataset by modifying the pointpillar.yaml file with DENSE_HEAD: NAME: AnchorHeadMulti and added RPN_HEAD_CFGS based on the three object categories in the KITTI dataset. The model trained successfully and was converted into a TensorRT engine, but it doesn't run correctly.

The output time, IDs and scores are confusing, and the data in the demo_boxes.txt output file are all zeros.Here is an example output:

------------------------------------
Module        Time        
------------------------------------
Preprocess    8.81295  ms
Pfe           534.183  ms
Scatter       3.85605  ms
Backbone      80.4403  ms
Postprocess   8.5541   ms
Summary       636.175  ms
------------------------------------
id: 0 labels: 2
      scores: 0.071332
id: 1 labels: 2
      scores: 0.0552087
id: 2 labels: 0
      scores: 0.117978
id: 3 labels: 0
      scores: 0.111822
id: 4 labels: 0
      scores: 0.110663
id: 5 labels: 0
      scores: 0.108945
.......
id: 2476 labels: 0
      scores: 0.0462514
id: 2504 labels: 539768118
      scores: 0.161591
id: 2516 labels: 1685202208
      scores: 0.161591
id: 2528 labels: 16873488
      scores: 0.161591
id: 2544 labels: 17134530
      scores: 0.646273
id: 3157 labels: 6146
      scores: 0.63662
id: 3177 labels: 255
      scores: 0.5
id: 3229 labels: 7
      scores: 0.5
id: 3233 labels: 2
      scores: 0.5
id: 3529 labels: -128
      scores: 0.166667
id: 3541 labels: 6
      scores: 0.0416667
id: 3549 labels: 0
      scores: 1

demo_boxes.txt:

0 0 0 0 0 0 0 
0 0 0 0 0 0 0 
......
0 0 0 0 0 0 0 

Could you please advise on how to resolve these issues? Any guidance you can provide would be greatly appreciated.

Thank you for your help!

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from PointPillars
[ RUN ] PointPillars.build_model
../model/cbgs_pp_multihead_backbone.onnx
#----------------------------------------------------------------
Input filename: ../model/cbgs_pp_multihead_pfe.onnx
ONNX IR version: 0.0.6
Opset version: 12
Producer name: pytorch
Producer version: 1.7
Domain:
Model version: 0
Doc string:
#----------------------------------------------------------------
WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
#----------------------------------------------------------------
Input filename: ../model/cbgs_pp_multihead_backbone.onnx
ONNX IR version: 0.0.6
Opset version: 10
Producer name: pytorch
Producer version: 1.7
Domain:
Model version: 0
Doc string:
#----------------------------------------------------------------
unknown file: Failure
C++ exception with description "Driver error: " thrown in the test body.
[ FAILED ] PointPillars.build_model (9718 ms)
[----------] 1 test from PointPillars (9718 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (9718 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] PointPillars.build_model

1 FAILED TEST

Thanks for the nice work. I met this error when I tried on the inference part. Would you please help me?

我把out_labels和out_detections按顺序一个个输出来，发现0有时候对应车有时候对应人，1,2,3也都不对。请问有人能解答一下吗？
0
41.2032 14.3038 -1.47437 4.50437 1.94559 1.63165 -0.153508
1
12.2437 14.8986 -0.351534 4.54949 1.88161 1.63159 1.24538
2
-5.04675 17.457 -0.105937 5.25686 2.03512 1.95156 -1.05545
0
-6.78207 -28.2318 -1.11056 9.87344 2.89405 3.43961 1.47927
0
-6.80986 26.5052 0.877446 4.09989 2.91479 3.26444 -0.217492
0
9.53663 -10.7155 -1.62894 0.589162 3.18327 0.912035 -0.226208
1
-4.21296 24.5938 -0.131519 0.430917 2.00643 1.07343 1.37179
2
10.8363 -3.58328 -1.53543 0.575118 2.16364 1.07301 -0.0863243
3
-0.491242 24.1164 -0.269451 0.365871 1.92069 1.03336 -1.34201
4
5.36149 -26.1912 -2.10972 0.568812 3.08624 0.916287 -0.19608
0
3.35874 15.7612 -0.448826 0.624788 0.628967 1.71421 0.635941
1
3.62719 25.2403 0.230647 0.603434 0.634361 1.74347 -1.067
2
4.49232 25.3182 0.191952 0.6769 0.664291 1.77158 0.533681

你好， 我想问一下，pointpillar原本不就是30+ms嘛，为什么您这边用了tensorrt后依然耗时27.5ms呢，为什么效果没有那么明显呢？