happinesslz / epnet Goto Github PK

Hello! I am a student from Xi'an Jiaotong University. I appreciate your work very much, and want to do some research on it.
In Fig.3 of your paper, you visualized the image feature map, and I want to do the same. Is the code of visualization in this codebase? If not, can you provide the code or tell me how to visualization it?
I am looking forward to your reply! Thank you very much!

how to fix it?

Hi, thank you for sharing the code!

I am very interested in the experiment of SUN-RGBD in your work, but there is no relevant code and model for this part at present. When will this part of code be released?

ImportError: /data4/shihaoWang/anaconda3/envs/EPNet/lib/python3.7/site-packages/iou3d-0.0.0-py3.7-linux-x86_64.egg/iou3d_cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe26detail37_typeMetaDataInstance_preallocated_32E
how should i deal with it?

Dear author: could you tell me how to test the unabled data? now i only can success run the eval mode, but failed on test mode, which special settings should we notice?

Thank you for publishing the great work!

I want to know how to visualize the Image Feature on Fig. 4?

Is there a corresponding visualization code?

hello,i have a question in EPNet\tools\kitti_object_eval_python\rotate_iou_cpu.py line 329

When I run the program, it prompts me that there is an error in this line。
TypeError: 'function' object is not subscriptable
I want to know what the [] after the function means and whether there is a problem with this usage。
thank u !

When trying to build the 3 libraries (pointnet, iou3d and roipoool3d) I have the same error with an invalid command for CUDA. Might it be from the version of the NVIDIA Toolkit?

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\nvcc -c src/ball_query_gpu.cu -o build\temp.win-amd64-3.6\Release\src/ball_query_gpu.obj -IC:\ProgramData\Anaconda3\envs\epnet\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\epnet\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\epnet\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\epnet\lib\site-packages\torch\include\THC "-IC:\Program Files" -IC:\ProgramData\Anaconda3\envs\epnet\include -IC:\ProgramData\Anaconda3\envs\epnet\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" -Xcompiler /EHsc -Xcompiler /wd4819 -Xcompiler /MD -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=pointnet2_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75
error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe\r\nC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\nvcc' failed: Invalid argument

I find that your evaluation metric is AP@recall 40. But many papers use AP@recall 11. Where should I change the evaluation code to control the evaluation metric ?

Hi,
Thanks for your great work! When I try to run bash build_and_install.sh to install packages, it shows RuntimeError: Error compiling objects for extension. My environment is python 3.7.4, torch 1.10.0, cuda 11.3. Could you please help me to figure out this error? Thanks!

Hi @happinesslz
Can you release your code based on Pointpillars?
I want to compare with my Pointpillars in my application scenario.
Many Thanks!

Whenever i wish to compile the bash script to install the roi3dpool etc packages i get the error that gcc has failed with exit code 1. I checked some pointrcnn files to which they said that this is being caused due to deprecation of certain cpp packages. Is there any hack to get it to work on ubuntu 20.04, gcc-9 and cuda 11.2?

Looking forward to the final version of the open source code.

I meet a question!help me!

Hello @happinesslz , thanks for the great code. I would like to detect multiple classes using EPNet. I modified your code based on the modifications in #3 except the last one. For the last one, I changed

Hello,
Every time I try to run the train_rcnn.py script it starts training but after a while I get an index error in the function sample_rois_for_rcnn from file proposal_target_layer.py with different index values.

while cur_gt[k].sum() == 0:
IndexError: index -11 is out of bounds for dimension 0 with size 10

Thank you very much for your excellent work and generosity in sharing the code. In the process of using the code, I found that the model and code provided by you could not reach the accuracy of the table. May I ask why?

Hello, i have a question about image size. SUNRGBD have different image size. How you deal with them. Can you tell me the details. Thank you very much!

Thanks for your contribution on 3D detection! It's such a good work
And I am interested in your model on SUNRGBD
Can you please provide more details about how to trian your model on SUNRGBD?
Thanks in advance!!! ^_^

Hi! I'm learning EPNet recently, your job is great and it is very helpful to me.
I have two questions about the code.

1. In kitti_rcnn_dataset.py, I only find the point cloud has data augmentation but image has not data augmentation, can they align?
2. In kitti_rcnn_dataset.py, if cfg.LI_FUSION.ENABLED is true, I think point cloud and img are both loaded by the method of self.get_rpn_with_li_fusion, and we don't need to use the method of self.get_rpn_sample, am I right?
   Look forward to your answers! Thank you very much!

Hello, I have some error while running build_and_install.sh. Maybe there is someone who already solved them.

• ValueError: underlying buffer has been detached
• src/iou3d.cpp(103): error C2131: expression did not evaluate to a constant
  Thank you in advance

I want to know why the scores are not in the interval (0.0,1.0), some of them even bigger than 10.

Hello, why should car and van be considered as one class and pedestrian and person_sitting as one class when separating training samples.

Hi, guys
I tried to train EPNet on 3090, I did the same thing just as the author told, and the environment is right.
But I failed to train, the training process will shutdown.
Oh, I do this on my 3090, is there a problem?

Could anybody give some advice?
Thanks a lot.

When can you release EPNet++ code?

i want to to know where the specific code for CE loss is located

Hi, thank you for sharing the code!
I have a small question about the data augmentation process. I wonder during the flipping and rotation of the lidar data, do you perform similar augmentation on the image? I think it is hard to flip the image along the forward direction...

Hi @happinesslz
I want to change your code to detect four class ,but got some errors,
I changed :
cfgs/LI_Fusion_with_attention_use_ce_loss.yaml, line1: CLASSES: Multi_class
kitti_rcnn_dataset.py , line45 add :
if classes == 'Multi_class':
self.classes = ('Background', 'Car', 'Pedestrian', 'Cyclist', 'Van')
aug_scene_root_dir = os.path.join(root_dir, 'KITTI', 'aug_scene_multi_class')

got such error:
File "/home/data/lidar_camera_fusion/EPNet/tools/train_rcnn.py", line 279, in
lr_scheduler_each_iter = (cfg.TRAIN.OPTIMIZER == 'adam_onecycle')
File "/home/data/lidar_camera_fusion/EPNet/tools/../tools/train_utils/train_utils.py", line 197, in train
loss, tb_dict, disp_dict = self._train_it(batch)
File "/home/data/lidar_camera_fusion/EPNet/tools/../tools/train_utils/train_utils.py", line 130, in _train_it
loss, tb_dict, disp_dict = self.model_fn(self.model, batch)
File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 81, in model_fn
rcnn_loss = get_rcnn_loss(model, ret_dict, tb_dict)
File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 204, in get_rcnn_loss
batch_loss_cls = F.binary_cross_entropy(torch.sigmoid(rcnn_cls_flat), cls_label, reduction='none')
File "/home/dming/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/functional.py", line 2070, in binary_cross_entropy
"!= input nelement ({})".format(target.numel(), input.numel()))
ValueError: Target and input must have the same number of elements. target nelement (64) != input nelement (320)

Have you done this job that detect four class ? or give me some advice.
Many Thanks !!!

Hello, is there a way to plot the predicted 3D bounding-boxes in the images after evaluating the trained model?
I tried to run 'vis_img.py', but it didn't have the expected result
Thank you for your answer

Hi when using my own dataset, I got the following assertion error. Could you explain why putting an assertion here?

Traceback (most recent call last):
File "/lbai/EPNet6000/tools/train_rcnn.py", line 276, in
lr_scheduler_each_iter = (cfg.TRAIN.OPTIMIZER == 'adam_onecycle')
File "/lbai/EPNet6000/tools/../tools/train_utils/train_utils.py", line 197, in train
loss, tb_dict, disp_dict = self._train_it(batch)
File "/lbai/EPNet6000/tools/../tools/train_utils/train_utils.py", line 130, in _train_it
loss, tb_dict, disp_dict = self.model_fn(self.model, batch)
File "/lbai/EPNet6000/tools/../lib/net/train_functions.py", line 50, in model_fn
ret_dict = model(input_data)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/lbai/EPNet6000/tools/../lib/net/point_rcnn.py", line 59, in forward
rois, roi_scores_raw = self.rpn.proposal_layer(rpn_scores_raw, rpn_reg, backbone_xyz) # (B, M, 7)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/lbai/EPNet6000/tools/../lib/rpn/proposal_layer.py", line 47, in forward
order_single)
File "/lbai/EPNet6000/tools/../lib/rpn/proposal_layer.py", line 93, in distance_based_proposal
assert i == 2, '%d' % i
AssertionError: 1

Process finished with exit code 1

Hello, I find your code "class Fusion_Conv" concact point_features and img_features directly. Where is your "w" part in your code?

I got a issue when run run_eval_model.sh

===================================
ENV INFO:
Ubuntu 18.04
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)
Python: 3.6.9
torch: 1.2.0
CUDA Version 10.0.130

===================================
error message like below:

$ bash run_eval_model.sh 
Traceback (most recent call last):
  File "eval_rcnn.py", line 7, in <module>
    from lib.net.point_rcnn import PointRCNN
  File "/home/ubuntu/ShareFiles/xieyi/epnet/EPNet-master/tools/../lib/net/point_rcnn.py", line 3, in <module>
    from lib.net.rpn import RPN
  File "/home/ubuntu/ShareFiles/xieyi/epnet/EPNet-master/tools/../lib/net/rpn.py", line 4, in <module>
    from lib.rpn.proposal_layer import ProposalLayer
  File "/home/ubuntu/ShareFiles/xieyi/epnet/EPNet-master/tools/../lib/rpn/proposal_layer.py", line 6, in <module>
    import lib.utils.iou3d.iou3d_utils as iou3d_utils
  File "/home/ubuntu/ShareFiles/xieyi/epnet/EPNet-master/tools/../lib/utils/iou3d/iou3d_utils.py", line 2, in <module>
    import iou3d_cuda
ImportError: /home/ubuntu/ShareFiles/xieyi/epnet/EPNet-master/py36/lib/python3.6/site-packages/iou3d-0.0.0-py3.6-linux-x86_64.egg/iou3d_cuda.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN2at19NonVariableTypeMode10is_enabledEv

非常感谢您的开源代码，我有一些问题想要咨询
1.Why is it important to ensure that the number of points sampled is equal to the number of points that can be projected onto the rgb img？

2How to understand this part of the code？

3 is it necessary to normalize xy before LI_fusion?

Dear author: you mention that you use three data augmentation strategies include: rotate, flip, and scale transfer the point clouds. But, in your code, you look like have not do the same augmentation for the input image, will there occur wrong when you fusion the pc with img?

I noticed that during training, the train_mode is set to rcnn_online. It looks for me the authors choose train the rpn and rcnn networks end-to-end. Therefore, I am wondering why. Obviously, end-to-end training is easier but is it better than training in two steps?

Thanks for your nice work. I have tested your code, I found it is very slow. However, you mentioned that the inferring speed is 10 FPS. Is it really 10 FPS?

When trying to run 'train_rcnn.py' I have the following KeyError from loading the images:
"train_functions.py", line 40, in model_fn
img = torch.from_numpy(data['img']).cuda(non_blocking=True).float().permute((0, 3, 1, 2))
KeyError: 'img'
I checked the keys and there is no key for 'img'
The keys are: dict_keys(['sample_id', 'random_select', 'rgb', 'aug_method', 'pts_input', 'pts_rect', 'pts_features', 'rpn_cls_label', 'rpn_reg_label', 'gt_boxes3d'])

I looked into the code but I could not find a solution.
Thank you in advance

Hi, I got the following error when training with multiple GPUs, how should I fix it?

raceback (most recent call last):
File "train_rcnn.py", line 270, in
trainer.train(
File "/root/autodl-tmp/EPNet/tools/../tools/train_utils/train_utils.py", line 197, in train
loss, tb_dict, disp_dict = self._train_it(batch)
File "/root/autodl-tmp/EPNet/tools/../tools/train_utils/train_utils.py", line 130, in _train_it
loss, tb_dict, disp_dict = self.model_fn(self.model, batch)
File "/root/autodl-tmp/EPNet/tools/../lib/net/train_functions.py", line 46, in model_fn
ret_dict = model(input_data)
File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 178, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
output.reraise()
File "/root/miniconda3/lib/python3.8/site-packages/torch/_utils.py", line 434, in reraise
raise exception
AssertionError: Caught AssertionError in replica 1 on device 1.
Original Traceback (most recent call last):
File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
output = module(*input, **kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/EPNet/tools/../lib/net/point_rcnn.py", line 52, in forward
rois, roi_scores_raw = self.rpn.proposal_layer(rpn_scores_raw, rpn_reg, backbone_xyz) # (B, M, 7)
File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/EPNet/tools/../lib/rpn/proposal_layer.py", line 46, in forward
scores_single, proposals_single = self.distance_based_proposal(scores_single, proposals_single,
File "/root/autodl-tmp/EPNet/tools/../lib/rpn/proposal_layer.py", line 93, in distance_based_proposal
assert i == 2, '%d' % i
AssertionError: 1

when I use "sh build_and_install.sh" but home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/group_points.o
c++ -MMD -MF /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/group_points.o.d -pthread -B /home/dxy/miniconda3/envs/openmmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/TH -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/dxy/miniconda3/envs/openmmlab/include/python3.7m -c -c /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/group_points.cpp -o /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/group_points.o -g -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pointnet2_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/group_points.cpp:5:10: fatal error: THC/THC.h: 没有那个文件或目录
5 | #include <THC/THC.h>
| ^~~~~~~~~~~
compilation terminated.
[2/4] c++ -MMD -MF /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/interpolate.o.d -pthread -B /home/dxy/miniconda3/envs/openmmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/TH -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/dxy/miniconda3/envs/openmmlab/include/python3.7m -c -c /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/interpolate.cpp -o /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/interpolate.o -g -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pointnet2_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/interpolate.o
c++ -MMD -MF /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/interpolate.o.d -pthread -B /home/dxy/miniconda3/envs/openmmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/TH -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/dxy/miniconda3/envs/openmmlab/include/python3.7m -c -c /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/interpolate.cpp -o /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/interpolate.o -g -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pointnet2_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/interpolate.cpp:3:10: fatal error: THC/THC.h: 没有那个文件或目录
3 | #include <THC/THC.h>
| ^~~~~~~~~~~
compilation terminated.
[3/4] c++ -MMD -MF /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/ball_query.o.d -pthread -B /home/dxy/miniconda3/envs/openmmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/TH -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/dxy/miniconda3/envs/openmmlab/include/python3.7m -c -c /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/ball_query.cpp -o /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/ball_query.o -g -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pointnet2_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/ball_query.o
c++ -MMD -MF /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/ball_query.o.d -pthread -B /home/dxy/miniconda3/envs/openmmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/TH -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/dxy/miniconda3/envs/openmmlab/include/python3.7m -c -c /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/ball_query.cpp -o /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/ball_query.o -g -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pointnet2_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/ball_query.cpp:3:10: fatal error: THC/THC.h: 没有那个文件或目录
3 | #include <THC/THC.h>
| ^~~~~~~~~~~
compilation terminated.
[4/4] c++ -MMD -MF /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/sampling.o.d -pthread -B /home/dxy/miniconda3/envs/openmmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/TH -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/dxy/miniconda3/envs/openmmlab/include/python3.7m -c -c /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/sampling.cpp -o /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/sampling.o -g -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pointnet2_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/sampling.o
c++ -MMD -MF /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/sampling.o.d -pthread -B /home/dxy/miniconda3/envs/openmmlab/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/TH -I/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/dxy/miniconda3/envs/openmmlab/include/python3.7m -c -c /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/sampling.cpp -o /home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/build/temp.linux-x86_64-cpython-37/src/sampling.o -g -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pointnet2_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/home/dxy/deepl/EPNet/pointnet2_lib/pointnet2/src/sampling.cpp:4:10: fatal error: THC/THC.h: 没有那个文件或目录
4 | #include <THC/THC.h>
| ^~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1906, in _run_ninja_build
env=env)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "setup.py", line 22, in
cmdclass = { 'build_ext': BuildExtension }
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/init.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
self.do_egg_install()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
self.run_command('bdist_egg')
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
self.run_command(cmdname)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
self.run_command('build_ext')
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
self.build_extensions()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
build_ext.build_extensions(self)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 468, in build_extensions
self._build_extensions_serial()
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 494, in _build_extensions_serial
self.build_extension(ext)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
_build_ext.build_extension(self, ext)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/Cython/Distutils/build_ext.py", line 127, in build_extension
super(build_ext, self).build_extension(ext)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 556, in build_extension
depends=ext.depends,
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 668, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1578, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/home/dxy/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1916, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

Thanks for your excellent job. I simply modified the CLASSES from "Car" to "Pedestrian" in the
config file "LI_Fusion_with_attention_use_ce_loss.yaml ". But the validation results become 0(both mAP and recall). I am wondering why? Is there any constraint about the class?

When I start the train_rcnn.py, I get the following error, can you tell me what the reason is：

File "train_rcnn.py", line 275, in
lr_scheduler_each_iter = (cfg.TRAIN.OPTIMIZER == 'adam_onecycle')
File "D:\Anaconda3\EPNet\tools..\tools\train_utils\train_utils.py", line 197, in train
loss, tb_dict, disp_dict = self._train_it(batch)
File "D:\Anaconda3\EPNet\tools..\tools\train_utils\train_utils.py", line 130, in _train_it
loss, tb_dict, disp_dict = self.model_fn(self.model, batch)
File "D:\Anaconda3\EPNet\tools..\lib\net\train_functions.py", line 37, in model_fn
img = torch.from_numpy(data['img']).cuda(non_blocking = True).float().permute((0, 3, 1, 2))
KeyError: 'img'

Hi! I'm learning EPNet recently, your job is great and it is very helpful to me.
I can't understand the function of cfg.RPN.USE_RGB. It doesn't appear in LI_Fusion_with_attention_use_ce_loss.yaml, but appear in other files. I guess cfg.RPN.USE_RGB is used to control whether to use image input.Am I right?
Thanks for your answer!

Hi,can I have your email or wechat. I have some question. thank you.

I am confused about the ablation experiment in Table 1.

If I understand it correctly, when LI-Fusion and CE are removed, the network structure will become very similar to PointRCNN. But the performance of the first row in Table 1 is much lower than that of PointRCNN. Why?

Paper: EPNet (without LI-Fusion and CE) 86.34, 77.52, 75.96
Code: EPNet (without LI-Fusion and CE) 88.76, 78.03, 76.20 (@r40)

Paper: PointRCNN 89.19, 78.85, 77.91
Code: PointRCNN 91.82, 80.66, 78.16 (@r40)