Hi!
Thanks for your great work!

When I try to train the 1-st model, Transfusion-L，I noticed that you metioned the copy-and-paste fade strategy. I wonder to know how I can achieve this mothod. I will appreciate it if you can give an example.

Hope for your response!
Best

Hello, when I download the fusion_voxel0075_R50.pth you provided, and run sh . /tools/dist_train.sh . /configs/MSMDFusion_nusc_voxel_LC.py 2 for the 2-nd stage training, the error is reported as follows, tried some solutions on the Internet still did not get a solution, I hope you can point out, thank you!

2023-09-14 10:43:15,801 - mmdet - INFO - Start running, host: xzluo@b5163d5d11c9, work_dir: /public/home/xzluo/zc/MSMDFusion-main/work_dirs/MSMDFusion_nusc_voxel_LC
2023-09-14 10:43:15,801 - mmdet - INFO - workflow: [('train', 1)], max: 6 epochs
Traceback (most recent call last):
File "./tools/train.py", line 283, in
main()
File "./tools/train.py", line 272, in main
train_detector(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter
outputs = self.model.train_step(data_batch, self.optimizer,
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/parallel/distributed.py", line 46, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 534, in forward_train
losses_pts = self.forward_pts_train(pts_feats, img_feats, gt_bboxes_3d,
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 574, in forward_pts_train
losses = self.pts_bbox_head.loss(*loss_inputs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 164, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/dense_heads/transfusion_head.py", line 1260, in loss
layer_loss_cls = self.loss_cls(layer_cls_score, layer_labels, layer_label_weights, avg_factor=max(num_pos, 1))
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/losses/focal_loss.py", line 170, in forward
loss_cls = self.loss_weight * calculate_loss_func(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/losses/focal_loss.py", line 85, in sigmoid_focal_loss
loss = _sigmoid_focal_loss(pred.contiguous(), target, gamma, alpha, None,
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/ops/focal_loss.py", line 54, in forward
ext_module.sigmoid_focal_loss_forward(
RuntimeError: SigmoidFocalLoss is not compiled with GPU support
Traceback (most recent call last):
File "./tools/train.py", line 283, in
main()
File "./tools/train.py", line 272, in main
train_detector(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter
outputs = self.model.train_step(data_batch, self.optimizer,
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/parallel/distributed.py", line 46, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 534, in forward_train
losses_pts = self.forward_pts_train(pts_feats, img_feats, gt_bboxes_3d,
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 574, in forward_pts_train
losses = self.pts_bbox_head.loss(*loss_inputs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 164, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/dense_heads/transfusion_head.py", line 1260, in loss
layer_loss_cls = self.loss_cls(layer_cls_score, layer_labels, layer_label_weights, avg_factor=max(num_pos, 1))
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/losses/focal_loss.py", line 170, in forward
loss_cls = self.loss_weight * calculate_loss_func(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/losses/focal_loss.py", line 85, in sigmoid_focal_loss
loss = _sigmoid_focal_loss(pred.contiguous(), target, gamma, alpha, None,
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/ops/focal_loss.py", line 54, in forward
ext_module.sigmoid_focal_loss_forward(
RuntimeError: SigmoidFocalLoss is not compiled with GPU support
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 29983) of binary: /public/home/xzluo/anaconda3/envs/zc/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 3/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group

How did you evaluate tracking, do you use CenterPoint? If so does not CenterPoint want the checkpoint files to be JSON format instead of pth, how did you manage that?

Im using this command:

(msmd) henrik@henrik-XPS-15-9510:~/CenterPoint$ python tools/nusc_tracking/pub_test.py --work_dir configs/nusc/voxelnet --checkpoint checkpoint/MSMDFusion.pth
======
Loading NuScenes tables for version v1.0-trainval...
23 category,
8 attribute,
4 visibility,
64386 instance,
12 sensor,
10200 calibrated_sensor,
2631083 ego_pose,
68 log,
850 scene,
34149 sample,
2631083 sample_data,
1166187 sample_annotation,
4 map,
Done loading in 21.414 seconds.
======
Reverse indexing ...
Done reverse indexing in 5.6 seconds.
======
Deploy OK
Use hungarian: False
Traceback (most recent call last):
  File "tools/nusc_tracking/pub_test.py", line 190, in <module>
    main()
  File "tools/nusc_tracking/pub_test.py", line 84, in main
    predictions=json.load(f)['results']
  File "/home/henrik/miniconda3/envs/msmd/lib/python3.7/json/__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/henrik/miniconda3/envs/msmd/lib/python3.7/json/__init__.py", line 343, in loads
    s = s.decode(detect_encoding(s), 'surrogatepass')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte

Should I convert MSMDFusion checkpoint files to JSON format or should i change in CenterPoint file pub_test to handle pth files?

Hi!

I noticed the nms_type is none in config file. I wonder whether you have done the nms part during the 3Dbboxes generating part.

wish for your response

I find the 3d_box_dim is 9. I konw the meaning of the first 7 values. I wonder what the last 2 values represent.

How much GPU memory is need to evaluate the model?

I´m running:python3 tools/test.py configs/MSMDFusion_nusc_voxel_LC.py checkpoint/MSMDFusion.pth --eval bbox

But get the error message: RuntimeError: CUDA out of memory. Tried to allocate 1.21 GiB (GPU 0; 7.77 GiB total capacity; 4.40 GiB already allocated; 1.21 GiB free; 5.05 GiB reserved in total by PyTorch)

Could you provide the information above? Such that i can tell whether if can run on my machine? Thanks a lot..

Hi!

https://github.com/SxJyJay/MSMDFusion/blame/7b5b2741e693ba8007c95e3e8951e4e67fbc47ed/configs/transfusion_nusc_voxel_L.py#L244-L244C10

1. In this line, you mention that the value of samples_per_gpu is 2, but in the previous part this value is set to 4. I am a little confused by this.
2. When I set the workers_per_gpu to 0 as the code does, I find that the utilization rate of my gpus (8 RTX 4090) is low and it must spend 14 days to train the model, is that nomal?
3. Due to the second problem, I set workers_per_gpu to 16 to make better use of my gpus. Besides, the value of samples_per_gpu is 4. After training,I get the following result:
   pts_bbox_NuScenes/NDS: 0.6944, pts_bbox_NuScenes/mAP: 0.6464
   which is a little bit lower than yours. Is this normal？If not, is that because of the change of workers_per_gpu?

Hope for yous reply!
Best

Hi，I notice that you use resnet50+fpn to generate image feature，but use centernet2 to generate virtual point，so I just wonder wether the inference speed 2.1HZ include the centernet2 inference time and virtual point generation time？if not，after add that time，how is the inference speed？

Hi!
I have 2 questions:

1. I wonder why this code is needed.
2. Besides I noticed that the original vitual point has 15 features, what these features are?

Hope for your response.
Best

Hello, thank you for your outstanding work. I am currently retrying your code.

The problem I encountered is when visualizing the demo and displaying this error:
ImportError: cannot import name 'init_ detector' from 'mmdet3d.apis' (/home/ps/yuji/MSMDFusion/mmdetection3d/mmdet3d/apis/init.py)

My command line is: Python demo/pcd_ demo.py demo/kitti_ 000008.bin configs/second/hv_ second_ secfpn_ 6x8_ 80e_ kitti-3d-car.py checkpoints/hv_ second_ secfpn_ 6x8_ 80e_ kitti-3d-car_ 20200620_ 230238-393f000c.pth --device 0 --out-dir /home/ps/yuji/MSMDFusion/outdir

According to the error report, it is due to an error in the file path of the import.

So I have two questions:
1.What is the difference between the mmdet3d folder and mmdetection3d?
2. If the visualized demo refers to the pcd_demo located in the mmdet3d folder, why is there an error that cannot be imported in mmdetection3d folder?

Looking forward to your reply. Thank you very much~

Hi, Thanks for your open-source code, I'm trying to reappear your code.
But I encountered some problems, I successfully installed the environment according to#Issues 1, and also processed the data. But when I run it, it shows_**ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject**_
I searched for related solutions, then improved the numpy version. But mmdet3d 0.11.0 requires numpy<=1.20.0. I tried to improve the numpy version and run it, but there was a problem with the dataset.

Original Traceback (most recent call last):

I think it is the reason for the numpy version. Could you please provide some advice? Thanks a lot!

ys.platform: linux
Python: 3.8.16 (default, Mar 2 2023, 03:21:46) [GCC 11.2.0]
CUDA available: True
GPU 0,1: NVIDIA GeForce RTX 3090
CUDA_HOME: /home/lxt/cuda-11.1
NVCC: Build cuda_11.1.TC455_06.29190527_0
GCC: gcc (Ubuntu 5.5.0-12ubuntu1) 5.5.0 20171010
PyTorch: 1.9.0+cu111
PyTorch compiling details: PyTorch built with:

• GCC 7.3
• C++ Version: 201402
• Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v2.1.2 (Git Hash 98be7e8afa711dc9b66c8ff3504129cb82013cdb)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 11.1
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
• CuDNN 8.0.5
• Magma 2.5.2
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.10.0+cu111
OpenCV: 4.7.0
MMCV: 1.3.18
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.1
MMDetection: 2.10.0
MMDetection3D: 0.11.0+

I can't understand Flc, where does it come from？

Thanks for your error report and we appreciate it a lot.

Checklist

1. I have searched related issues but cannot get the expected help.
2. The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.

Reproduction

1. What command or script did you run?

sh ./tools/dist_test.sh ./configs/MSMDFusion_nusc_voxel_LC.py $ckpt_path$ 8 --eval bbox
A placeholder for the command.

2. Did you make any modifications on the code or config? Did you understand what you have modified?
   No change to the code
3. What dataset did you use?
   Nuscense dataset

Environment

1. Please run python mmdet3d/utils/collect_env.py to collect necessary environment infomation and paste it here.
   ImportError: cannot import name 'get_git_hash' from 'mmcv.utils' (/home/cc7287/mambaforge/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/init.py)
2. You may add addition that may be helpful for locating the problem, such as
   • How you installed PyTorch [e.g., pip, conda, source]
   • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)
     conda create -n open-mmlab python=3.7 -y
     conda activate open-mmlab

install latest pytorch prebuilt with the default prebuilt CUDA version (usually the latest)

conda install -c pytorch pytorch torchvision -y

install mmcv

pip install mmcv-full

install mmdetection

pip install git+https://github.com/open-mmlab/mmdetection.git

install mmdetection3d

git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
pip install -v -e .

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Describe the bug
Hi I was wondering if someone else had issues when building the docker, I get compatibility issues.

Reproduction

1. What command or script did you run?

(base) henrik@henrik-XPS-15-9510:~/mmdetection/mmdetection3d/MSMDFusion/docker$ docker build -t msmd .

2. Did you make any modifications on the code or config? Did you understand what you have modified?
   Yes I modified the MSMDFusion docker file. Added the follwoing in row 11 to fix earlier errors. I also specified mmcv-full to be 1.3.0 see below.

To fix GPG key error when running apt-get update

RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub
---------------------------------------------------Now the docker file looks like this-------------------------------------------------------------------

ARG PYTORCH="1.6.0"
ARG CUDA="10.1"
ARG CUDNN="7"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel

ENV TORCH_CUDA_ARCH_LIST="6.0 6.1 7.0+PTX"
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"
ENV CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"

### To fix GPG key error when running apt-get update
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub

RUN apt-get update && apt-get install -y ffmpeg libsm6 libxext6 git ninja-build libglib2.0-0 libsm6 libxrender-dev libxext6 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install MMCV
RUN pip install mmcv-full==1.3.0
RUN pip install mmdet

# Install MMDetection
RUN conda clean --all
RUN git clone https://github.com/open-mmlab/mmdetection3d.git /mmdetection3d
WORKDIR /mmdetection3d
ENV FORCE_CUDA="1"
RUN pip install -r requirements/build.txt
RUN pip install --no-cache-dir -e .

# uninstall pycocotools installed by nuscenes-devkit and reinstall mmpycocotools
RUN pip uninstall pycocotools --no-cache-dir -y
RUN pip install mmpycocotools --no-cache-dir --force --no-deps

3. What dataset did you use?

I downloaded the nuscenes dataset.

1. Please run python mmdet3d/utils/collect_env.py to collect necessary environment infomation and paste it here:
   (open-mmlab) henrik@henrik-XPS-15-9510:/mmdetection/mmdetection3d/mmdet3d/utils$ python collect_env.py
   sys.platform: linux
   Python: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0]
   CUDA available: True
   MUSA available: False
   numpy_random_seed: 2147483648
   GPU 0: NVIDIA GeForce RTX 3050 Ti Laptop GPU
   CUDA_HOME: /home/henrik/miniconda3/envs/open-mmlab
   NVCC: Cuda compilation tools, release 11.5, V11.5.119
   GCC: gcc (Ubuntu 11.4.0-1ubuntu122.04) 11.4.0
   PyTorch: 2.0.0
   PyTorch compiling details: PyTorch built with:

• GCC 9.3
• C++ Version: 201703
• Intel(R) oneAPI Math Kernel Library Version 2023.1-Product Build 20230303 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 11.8
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_37,code=compute_37
• CuDNN 8.7
• Magma 2.6.1
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.15.0
OpenCV: 4.9.0
MMEngine: 0.10.3
MMDetection: 3.3.0
MMDetection3D: 1.4.0+fe25f7a
spconv2.0: False
3. You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

base) henrik@henrik-XPS-15-9510:~/mmdetection/mmdetection3d/MSMDFusion/docker$ docker build -t msmd .
[+] Building 92.3s (14/16)                                                                                                                                                                   docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                   0.0s
 => => transferring dockerfile: 1.29kB                                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/pytorch/pytorch:1.6.0-cuda10.1-cudnn7-devel                                                                                                                 0.9s
 => [internal] load .dockerignore                                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                                        0.0s
 => [ 1/13] FROM docker.io/pytorch/pytorch:1.6.0-cuda10.1-cudnn7-devel@sha256:ccebb46f954b1d32a4700aaeae0e24bd68653f92c6f276a608bf592b660b63d7                                                         0.0s
 => CACHED [ 2/13] RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub                                                                0.0s
 => CACHED [ 3/13] RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub                                                    0.0s
 => CACHED [ 4/13] RUN apt-get update && apt-get install -y ffmpeg libsm6 libxext6 git ninja-build libglib2.0-0 libsm6 libxrender-dev libxext6     && apt-get clean     && rm -rf /var/lib/apt/lists/  0.0s
 => CACHED [ 5/13] RUN pip install mmcv-full==1.3.0                                                                                                                                                    0.0s
 => CACHED [ 6/13] RUN pip install mmdet                                                                                                                                                               0.0s
 => CACHED [ 7/13] RUN conda clean --all                                                                                                                                                               0.0s
 => CACHED [ 8/13] RUN git clone https://github.com/open-mmlab/mmdetection3d.git /mmdetection3d                                                                                                        0.0s
 => CACHED [ 9/13] WORKDIR /mmdetection3d                                                                                                                                                              0.0s
 => CACHED [10/13] RUN pip install -r requirements/build.txt                                                                                                                                           0.0s
 => ERROR [11/13] RUN pip install --no-cache-dir -e .                                                                                                                                                 91.4s
------                                                                                                                                                                                                      
 > [11/13] RUN pip install --no-cache-dir -e .:                                                                                                                                                             
0.568 Obtaining file:///mmdetection3d                                                                                                                                                                       
1.645 Collecting lyft_dataset_sdk                                                                                                                                                                           
1.719   Downloading lyft_dataset_sdk-0.0.8-py2.py3-none-any.whl (32 kB)                                                                                                                                     
1.794 Collecting networkx>=2.5                                                                                                                                                                              
1.814   Downloading networkx-2.6.3-py3-none-any.whl (1.9 MB)
2.378 Collecting numba
2.395   Downloading numba-0.56.4-cp37-cp37m-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.5 MB)
2.891 Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from mmdet3d==1.4.0) (1.18.5)
2.931 Collecting nuscenes-devkit
2.949   Downloading nuscenes_devkit-1.1.11-py3-none-any.whl (313 kB)
4.186 Collecting open3d
3.327   Downloading open3d-0.13.0-cp37-cp37m-manylinux2014_x86_64.whl (300.6 MB)
44.41 Collecting plyfile
44.43   Downloading plyfile-0.9-py3-none-any.whl (30 kB)
44.56 Collecting scikit-image
44.58   Downloading scikit_image-0.19.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.5 MB)
45.93 Collecting tensorboard
45.95   Downloading tensorboard-2.11.2-py3-none-any.whl (6.0 MB)
46.82 Collecting trimesh
46.83   Downloading trimesh-4.1.3-py3-none-any.whl (690 kB)
46.98 Collecting cachetools>=3.1.0
46.99   Downloading cachetools-5.3.2-py3-none-any.whl (9.3 kB)
47.00 Requirement already satisfied: opencv-python>=3.4.2.17 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (4.9.0.80)
47.26 Collecting pandas
47.29   Downloading pandas-1.3.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
48.60 Collecting black
48.62   Downloading black-23.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
48.86 Collecting flake8
48.87   Downloading flake8-5.0.4-py2.py3-none-any.whl (61 kB)
48.97 Collecting pytest
48.99   Downloading pytest-7.4.4-py3-none-any.whl (325 kB)
49.12 Collecting plotly
49.14   Downloading plotly-5.18.0-py3-none-any.whl (15.6 MB)
51.57 Collecting pyquaternion>=0.9.5
51.59   Downloading pyquaternion-0.9.9-py3-none-any.whl (14 kB)
51.61 Requirement already satisfied: Pillow>=5.2.0 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (7.2.0)
51.62 Requirement already satisfied: matplotlib in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (3.5.3)
51.78 Collecting scikit-learn>=0.19.2
51.79   Downloading scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
54.80 Requirement already satisfied: Shapely>=1.6.4.post2 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (2.0.2)
54.84 Collecting fire
54.86   Downloading fire-0.5.0.tar.gz (88 kB)
55.05 Requirement already satisfied: scipy>=1.1.0 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (1.7.3)
55.05 Requirement already satisfied: tqdm>=4.25.0 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (4.46.0)
55.06 Requirement already satisfied: importlib-metadata; python_version < "3.9" in /opt/conda/lib/python3.7/site-packages (from numba->mmdet3d==1.4.0) (6.7.0)
55.18 Collecting llvmlite<0.40,>=0.39.0dev0
55.20   Downloading llvmlite-0.39.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.6 MB)
59.10 Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from numba->mmdet3d==1.4.0) (46.4.0.post20200518)
59.10 Requirement already satisfied: pycocotools>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from nuscenes-devkit->mmdet3d==1.4.0) (2.0.7)
59.17 Collecting descartes
59.19   Downloading descartes-1.1.0-py3-none-any.whl (5.8 kB)
59.26 Collecting ipywidgets>=7.6.0
59.27   Downloading ipywidgets-8.1.2-py3-none-any.whl (139 kB)
59.46 Collecting jupyter-packaging~=0.10
59.48   Downloading jupyter_packaging-0.12.3-py3-none-any.whl (15 kB)
59.53 Collecting pygments>=2.7.4
59.54   Downloading pygments-2.17.2-py3-none-any.whl (1.2 MB)
59.74 Collecting wheel>=0.36.0
59.75   Downloading wheel-0.42.0-py3-none-any.whl (65 kB)
59.85 Collecting pyyaml>=5.4.1
59.86   Downloading PyYAML-6.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (670 kB)
60.17 Collecting jupyterlab==3.*,>=3.0.0
60.19   Downloading jupyterlab-3.6.7-py3-none-any.whl (8.9 MB)
61.38 Requirement already satisfied: addict in /opt/conda/lib/python3.7/site-packages (from open3d->mmdet3d==1.4.0) (2.4.0)
61.44 Collecting imageio>=2.4.1
61.45   Downloading imageio-2.31.2-py3-none-any.whl (313 kB)
61.59 Collecting tifffile>=2019.7.26
61.60   Downloading tifffile-2021.11.2-py3-none-any.whl (178 kB)
61.71 Collecting PyWavelets>=1.1.1
61.74   Downloading PyWavelets-1.3.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.4 MB)
62.46 Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.7/site-packages (from scikit-image->mmdet3d==1.4.0) (23.2)
63.30 Collecting grpcio>=1.24.3
63.33   Downloading grpcio-1.60.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)
64.03 Collecting tensorboard-plugin-wit>=1.6.0
64.05   Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
64.19 Collecting absl-py>=0.4
64.20   Downloading absl_py-2.1.0-py3-none-any.whl (133 kB)
64.34 Collecting google-auth<3,>=1.6.3
64.35   Downloading google_auth-2.27.0-py2.py3-none-any.whl (186 kB)
64.43 Collecting markdown>=2.6.8
64.44   Downloading Markdown-3.4.4-py3-none-any.whl (94 kB)
64.49 Collecting tensorboard-data-server<0.7.0,>=0.6.0
64.50   Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
65.35 Collecting protobuf<4,>=3.9.2
65.37   Downloading protobuf-3.20.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
65.54 Collecting werkzeug>=1.0.1
65.57   Downloading Werkzeug-2.2.3-py3-none-any.whl (233 kB)
65.60 Requirement already satisfied: requests<3,>=2.21.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard->mmdet3d==1.4.0) (2.23.0)
65.65 Collecting google-auth-oauthlib<0.5,>=0.4.1
65.67   Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
65.68 Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from pandas->lyft_dataset_sdk->mmdet3d==1.4.0) (2.8.2)
65.68 Requirement already satisfied: pytz>=2017.3 in /opt/conda/lib/python3.7/site-packages (from pandas->lyft_dataset_sdk->mmdet3d==1.4.0) (2020.1)
65.76 Collecting typed-ast>=1.4.2; python_version < "3.8" and implementation_name == "cpython"
65.78   Downloading typed_ast-1.5.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (778 kB)
65.91 Requirement already satisfied: typing-extensions>=3.10.0.0; python_version < "3.10" in /opt/conda/lib/python3.7/site-packages (from black->lyft_dataset_sdk->mmdet3d==1.4.0) (4.7.1)
65.94 Collecting pathspec>=0.9.0
65.96   Downloading pathspec-0.11.2-py3-none-any.whl (29 kB)
65.99 Collecting mypy-extensions>=0.4.3
66.01   Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
66.05 Collecting click>=8.0.0
66.07   Downloading click-8.1.7-py3-none-any.whl (97 kB)
66.09 Requirement already satisfied: tomli>=1.1.0; python_version < "3.11" in /opt/conda/lib/python3.7/site-packages (from black->lyft_dataset_sdk->mmdet3d==1.4.0) (2.0.1)
66.11 Requirement already satisfied: platformdirs>=2 in /opt/conda/lib/python3.7/site-packages (from black->lyft_dataset_sdk->mmdet3d==1.4.0) (4.0.0)
66.13 Collecting pyflakes<2.6.0,>=2.5.0
66.15   Downloading pyflakes-2.5.0-py2.py3-none-any.whl (66 kB)
66.19 Collecting pycodestyle<2.10.0,>=2.9.0
66.20   Downloading pycodestyle-2.9.1-py2.py3-none-any.whl (41 kB)
66.24 Collecting mccabe<0.8.0,>=0.7.0
66.26   Downloading mccabe-0.7.0-py2.py3-none-any.whl (7.3 kB)
66.30 Collecting exceptiongroup>=1.0.0rc8; python_version < "3.11"
66.32   Downloading exceptiongroup-1.2.0-py3-none-any.whl (16 kB)
66.35 Collecting pluggy<2.0,>=0.12
66.36   Downloading pluggy-1.2.0-py3-none-any.whl (17 kB)
66.40 Collecting iniconfig
66.41   Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
66.45 Collecting tenacity>=6.2.0
66.46   Downloading tenacity-8.2.3-py3-none-any.whl (24 kB)
66.47 Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.7/site-packages (from matplotlib->lyft_dataset_sdk->mmdet3d==1.4.0) (4.38.0)
66.49 Requirement already satisfied: pyparsing>=2.2.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->lyft_dataset_sdk->mmdet3d==1.4.0) (3.1.1)
66.50 Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->lyft_dataset_sdk->mmdet3d==1.4.0) (1.4.5)
66.50 Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.7/site-packages (from matplotlib->lyft_dataset_sdk->mmdet3d==1.4.0) (0.11.0)
66.54 Collecting joblib>=0.11
66.56   Downloading joblib-1.3.2-py3-none-any.whl (302 kB)
66.63 Collecting threadpoolctl>=2.0.0
66.64   Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
66.65 Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from fire->lyft_dataset_sdk->mmdet3d==1.4.0) (1.14.0)
66.68 Collecting termcolor
66.70   Downloading termcolor-2.3.0-py3-none-any.whl (6.9 kB)
66.71 Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.9"->numba->mmdet3d==1.4.0) (3.15.0)
66.72 Requirement already satisfied: traitlets>=4.3.1 in /opt/conda/lib/python3.7/site-packages (from ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (4.3.3)
66.73 Requirement already satisfied: ipython>=6.1.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (7.16.1)
66.76 Collecting comm>=0.1.3
66.77   Downloading comm-0.1.4-py3-none-any.whl (6.6 kB)
66.84 Collecting widgetsnbextension~=4.0.10
66.86   Downloading widgetsnbextension-4.0.10-py3-none-any.whl (2.3 MB)
67.20 Collecting jupyterlab-widgets~=3.0.10
67.22   Downloading jupyterlab_widgets-3.0.10-py3-none-any.whl (215 kB)
67.30 Collecting tomlkit
67.32   Downloading tomlkit-0.12.3-py3-none-any.whl (37 kB)
67.35 Collecting deprecation
67.37   Downloading deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
67.42 Collecting jupyter-core
67.43   Downloading jupyter_core-4.12.0-py3-none-any.whl (89 kB)
67.60 Collecting jupyter-ydoc~=0.2.4
67.61   Downloading jupyter_ydoc-0.2.5-py3-none-any.whl (6.2 kB)
67.68 Collecting jupyterlab-server~=2.19
67.70   Downloading jupyterlab_server-2.24.0-py3-none-any.whl (57 kB)
67.85 Collecting jupyter-server-ydoc~=0.8.0
67.87   Downloading jupyter_server_ydoc-0.8.0-py3-none-any.whl (11 kB)
67.97 Collecting jupyter-server<3,>=1.16.0
67.98   Downloading jupyter_server-1.24.0-py3-none-any.whl (347 kB)
68.13 Collecting notebook<7
68.14   Downloading notebook-6.5.6-py3-none-any.whl (529 kB)
68.41 Collecting nbclassic
68.42   Downloading nbclassic-1.0.0-py3-none-any.whl (10.0 MB)
69.72 Requirement already satisfied: jinja2>=2.1 in /opt/conda/lib/python3.7/site-packages (from jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (2.11.2)
69.82 Collecting tornado>=6.1.0
69.84   Downloading tornado-6.2-cp37-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (423 kB)
69.95 Collecting rsa<5,>=3.1.4
69.96   Downloading rsa-4.9-py3-none-any.whl (34 kB)
70.02 Collecting pyasn1-modules>=0.2.1
70.03   Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
70.19 Collecting MarkupSafe>=2.1.1
70.21   Downloading MarkupSafe-2.1.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
70.21 Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->mmdet3d==1.4.0) (1.25.8)
70.23 Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->mmdet3d==1.4.0) (2.9)
70.23 Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->mmdet3d==1.4.0) (2020.6.20)
70.23 Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->mmdet3d==1.4.0) (3.0.4)
70.25 Collecting requests-oauthlib>=0.7.0
70.27   Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
70.29 Requirement already satisfied: ipython-genutils in /opt/conda/lib/python3.7/site-packages (from traitlets>=4.3.1->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.2.0)
70.29 Requirement already satisfied: decorator in /opt/conda/lib/python3.7/site-packages (from traitlets>=4.3.1->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (4.4.2)
70.29 Requirement already satisfied: pexpect; sys_platform != "win32" in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (4.8.0)
70.29 Requirement already satisfied: pickleshare in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.7.5)
70.29 Requirement already satisfied: backcall in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.2.0)
70.29 Requirement already satisfied: jedi>=0.10 in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.17.1)
70.30 Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (3.0.5)
70.50 Collecting y-py<0.7.0,>=0.6.0
70.52   Downloading y_py-0.6.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
70.81 Collecting jsonschema>=4.17.3
70.83   Downloading jsonschema-4.17.3-py3-none-any.whl (90 kB)
70.89 Collecting json5>=0.9.0
70.91   Downloading json5-0.9.14-py2.py3-none-any.whl (19 kB)
70.94 Collecting babel>=2.10
70.96   Downloading Babel-2.14.0-py3-none-any.whl (11.0 MB)
72.44 Collecting ypy-websocket<0.9.0,>=0.8.2
72.45   Downloading ypy_websocket-0.8.4-py3-none-any.whl (10 kB)
72.59 Collecting jupyter-server-fileid<1,>=0.6.0
72.60   Downloading jupyter_server_fileid-0.9.1-py3-none-any.whl (16 kB)
72.66 Collecting argon2-cffi
72.68   Downloading argon2_cffi-23.1.0-py3-none-any.whl (15 kB)
72.73 Collecting websocket-client
72.75   Downloading websocket_client-1.6.1-py3-none-any.whl (56 kB)
72.81 Collecting jupyter-client>=6.1.12
72.83   Downloading jupyter_client-7.4.9-py3-none-any.whl (133 kB)
72.91 Collecting terminado>=0.8.3
72.92   Downloading terminado-0.17.1-py3-none-any.whl (17 kB)
72.96 Collecting nbformat>=5.2.0
72.98   Downloading nbformat-5.8.0-py3-none-any.whl (77 kB)
73.38 Collecting pyzmq>=17
73.41   Downloading pyzmq-25.1.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)
73.61 Collecting Send2Trash
73.63   Downloading Send2Trash-1.8.2-py3-none-any.whl (18 kB)
73.68 Collecting prometheus-client
73.69   Downloading prometheus_client-0.17.1-py3-none-any.whl (60 kB)
73.76 Collecting nbconvert>=6.4.4
73.78   Downloading nbconvert-7.6.0-py3-none-any.whl (290 kB)
73.89 Collecting anyio<4,>=3.1.0
73.90   Downloading anyio-3.7.1-py3-none-any.whl (80 kB)
73.99 Collecting ipykernel
74.01   Downloading ipykernel-6.16.2-py3-none-any.whl (138 kB)
74.08 Collecting nest-asyncio>=1.5
74.09   Downloading nest_asyncio-1.6.0-py3-none-any.whl (5.2 kB)
74.11 Collecting notebook-shim>=0.2.3
74.13   Downloading notebook_shim-0.2.3-py3-none-any.whl (13 kB)
74.21 Collecting pyasn1>=0.1.3
74.22   Downloading pyasn1-0.5.1-py2.py3-none-any.whl (84 kB)
74.26 Collecting oauthlib>=3.0.0
74.28   Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
74.33 Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.7/site-packages (from pexpect; sys_platform != "win32"->ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.6.0)
74.33 Requirement already satisfied: parso<0.8.0,>=0.7.0 in /opt/conda/lib/python3.7/site-packages (from jedi>=0.10->ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.7.0)
74.33 Requirement already satisfied: wcwidth in /opt/conda/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.2.5)
74.38 Collecting pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0
74.39   Downloading pyrsistent-0.19.3-py3-none-any.whl (57 kB)
74.45 Collecting importlib-resources>=1.4.0; python_version < "3.9"
74.47   Downloading importlib_resources-5.12.0-py3-none-any.whl (36 kB)
74.51 Collecting pkgutil-resolve-name>=1.3.10; python_version < "3.9"
74.53   Downloading pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
74.56 Collecting attrs>=17.4.0
74.57   Downloading attrs-23.2.0-py3-none-any.whl (60 kB)
74.73 Collecting aiosqlite<1,>=0.17.0
74.74   Downloading aiosqlite-0.19.0-py3-none-any.whl (15 kB)
74.80 Collecting aiofiles<23,>=22.1.0
74.81   Downloading aiofiles-22.1.0-py3-none-any.whl (14 kB)
74.85 Collecting jupyter-events>=0.5.0
74.86   Downloading jupyter_events-0.6.3-py3-none-any.whl (18 kB)
74.90 Collecting argon2-cffi-bindings
74.91   Downloading argon2_cffi_bindings-21.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (86 kB)
74.94 Collecting entrypoints
74.96   Downloading entrypoints-0.4-py3-none-any.whl (5.3 kB)
74.99 Collecting fastjsonschema
75.00   Downloading fastjsonschema-2.19.1-py3-none-any.whl (23 kB)
75.05 Collecting nbclient>=0.5.0
75.06   Downloading nbclient-0.7.4-py3-none-any.whl (73 kB)
75.13 Collecting tinycss2
75.14   Downloading tinycss2-1.2.1-py3-none-any.whl (21 kB)
75.19 Collecting bleach!=5.0.0
75.21   Downloading bleach-6.0.0-py3-none-any.whl (162 kB)
75.27 Collecting mistune<4,>=2.0.3
75.28   Downloading mistune-3.0.2-py3-none-any.whl (47 kB)
75.31 Collecting defusedxml
75.33   Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
75.36 Collecting pandocfilters>=1.4.1
75.37   Downloading pandocfilters-1.5.1-py2.py3-none-any.whl (8.7 kB)
75.38 Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.7/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=1.16.0->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (4.9.1)
75.41 Collecting jupyterlab-pygments
75.43   Downloading jupyterlab_pygments-0.2.2-py2.py3-none-any.whl (21 kB)
75.46 Collecting sniffio>=1.1
75.47   Downloading sniffio-1.3.0-py3-none-any.whl (10 kB)
75.69 Collecting debugpy>=1.0
75.71   Downloading debugpy-1.7.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
76.07 Requirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from ipykernel->notebook<7->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (5.7.0)
76.10 Collecting matplotlib-inline>=0.1
76.12   Downloading matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
76.16 Collecting rfc3986-validator>=0.1.1
76.17   Downloading rfc3986_validator-0.1.1-py2.py3-none-any.whl (4.2 kB)
76.20 Collecting python-json-logger>=2.0.4
76.21   Downloading python_json_logger-2.0.7-py3-none-any.whl (8.1 kB)
76.24 Collecting rfc3339-validator
76.26   Downloading rfc3339_validator-0.1.4-py2.py3-none-any.whl (3.5 kB)
76.27 Requirement already satisfied: cffi>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=1.16.0->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (1.14.0)
76.29 Collecting webencodings>=0.4
76.31   Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
76.32 Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.7/site-packages (from beautifulsoup4->nbconvert>=6.4.4->jupyter-server<3,>=1.16.0->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (2.0.1)
76.32 Requirement already satisfied: pycparser in /opt/conda/lib/python3.7/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=1.16.0->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (2.20)
76.32 Building wheels for collected packages: fire
76.32   Building wheel for fire (setup.py): started
76.48   Building wheel for fire (setup.py): finished with status 'done'
76.48   Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116936 sha256=e71e38260e683cd2f241967c10362ed41096b703538c8eacc1df9483d439ca26
76.48   Stored in directory: /tmp/pip-ephem-wheel-cache-3gf77sm_/wheels/20/97/e1/dd2c472bebcdcaa85fdc07d0f19020299f1c86773028860c53
76.48 Successfully built fire
77.06 ERROR: flake8 5.0.4 has requirement importlib-metadata<4.3,>=1.1.0; python_version < "3.8", but you'll have importlib-metadata 6.7.0 which is incompatible.
77.06 ERROR: nuscenes-devkit 1.1.11 has requirement numpy>=1.22.0, but you'll have numpy 1.18.5 which is incompatible.
77.06 ERROR: nuscenes-devkit 1.1.11 has requirement Shapely<2.0.0, but you'll have shapely 2.0.2 which is incompatible.
77.06 ERROR: jupyter-packaging 0.12.3 has requirement setuptools>=60.2.0, but you'll have setuptools 46.4.0.post20200518 which is incompatible.
77.06 ERROR: nbformat 5.8.0 has requirement traitlets>=5.1, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: nbclient 0.7.4 has requirement traitlets>=5.3, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: nbconvert 7.6.0 has requirement jinja2>=3.0, but you'll have jinja2 2.11.2 which is incompatible.
77.06 ERROR: nbconvert 7.6.0 has requirement traitlets>=5.1, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: jupyter-server 1.24.0 has requirement traitlets>=5.1, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: jupyterlab-server 2.24.0 has requirement jinja2>=3.0.3, but you'll have jinja2 2.11.2 which is incompatible.
77.06 ERROR: jupyterlab-server 2.24.0 has requirement requests>=2.28, but you'll have requests 2.23.0 which is incompatible.
77.06 ERROR: jupyter-events 0.6.3 has requirement traitlets>=5.3, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: ipykernel 6.16.2 has requirement ipython>=7.23.1, but you'll have ipython 7.16.1 which is incompatible.
77.06 ERROR: ipykernel 6.16.2 has requirement traitlets>=5.1.0, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: notebook 6.5.6 has requirement pyzmq<25,>=17, but you'll have pyzmq 25.1.2 which is incompatible.
77.06 ERROR: open3d 0.13.0 has requirement pillow>=8.2.0, but you'll have pillow 7.2.0 which is incompatible.
77.06 ERROR: imageio 2.31.2 has requirement pillow>=8.3.2, but you'll have pillow 7.2.0 which is incompatible.
77.06 Installing collected packages: cachetools, pandas, typed-ast, pathspec, mypy-extensions, click, black, pyflakes, pycodestyle, mccabe, flake8, exceptiongroup, pluggy, iniconfig, pytest, tenacity, plotly, pyquaternion, joblib, threadpoolctl, scikit-learn, termcolor, fire, lyft-dataset-sdk, networkx, llvmlite, numba, descartes, nuscenes-devkit, comm, widgetsnbextension, jupyterlab-widgets, ipywidgets, tomlkit, wheel, deprecation, jupyter-packaging, pygments, pyyaml, jupyter-core, y-py, jupyter-ydoc, pyrsistent, importlib-resources, pkgutil-resolve-name, attrs, jsonschema, argon2-cffi-bindings, argon2-cffi, websocket-client, tornado, entrypoints, nest-asyncio, pyzmq, jupyter-client, terminado, fastjsonschema, nbformat, Send2Trash, prometheus-client, nbclient, MarkupSafe, webencodings, tinycss2, bleach, mistune, defusedxml, pandocfilters, jupyterlab-pygments, nbconvert, sniffio, anyio, jupyter-server, json5, babel, jupyterlab-server, aiosqlite, aiofiles, ypy-websocket, rfc3986-validator, python-json-logger, rfc3339-validator, jupyter-events, jupyter-server-fileid, jupyter-server-ydoc, debugpy, matplotlib-inline, ipykernel, notebook-shim, nbclassic, notebook, jupyterlab, open3d, plyfile, imageio, tifffile, PyWavelets, scikit-image, grpcio, tensorboard-plugin-wit, absl-py, pyasn1, rsa, pyasn1-modules, google-auth, markdown, tensorboard-data-server, protobuf, werkzeug, oauthlib, requests-oauthlib, google-auth-oauthlib, tensorboard, trimesh, mmdet3d
89.48   Attempting uninstall: wheel
89.48     Found existing installation: wheel 0.34.2
89.48     Uninstalling wheel-0.34.2:
89.83       Successfully uninstalled wheel-0.34.2
89.87   Attempting uninstall: pygments
89.87     Found existing installation: Pygments 2.6.1
89.91     Uninstalling Pygments-2.6.1:
90.01       Successfully uninstalled Pygments-2.6.1
90.37   Attempting uninstall: pyyaml
90.37     Found existing installation: PyYAML 5.3.1
90.91 ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
------
Dockerfile:29
--------------------
  27 |     ENV FORCE_CUDA="1"
  28 |     RUN pip install -r requirements/build.txt
  29 | >>> RUN pip install --no-cache-dir -e .
  30 |     
  31 |     # uninstall pycocotools installed by nuscenes-devkit and reinstall mmpycocotools
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install --no-cache-dir -e ." did not complete successfully: exit code: 1

Any help would be appreciated.

Hi, Thanks for your open-source code, I'm trying to reappear your code.
Although I complete the preprocess of nuscenes, I failed in training stage due to the unknown version of python library.
Could you provide the current version of cuda, python, pytorch, torch-scatter, mmcv-full, mmdetection, spconv-2 in MSMDFusion?
Thanks a lot!

2023-10-09 11:44:22,955 - mmdet - INFO - Exp name: MSMDFusion_nusc_voxel_LC.py
2023-10-09 11:44:22,956 - mmdet - INFO - Epoch [1][32000/32025] lr: 1.000e-04, eta: 3 days, 4:50:02, time: 1.717, data_time: 0.076, memory: 8875, loss_heatmap: 0.4555, layer_-1_loss_cls: 0.0766, layer_-1_loss_bbox: 0.4751, matched_ious: 0.5853, loss: 1.0072, grad_norm: 1.5121
2023-10-09 11:45:07,229 - mmdet - INFO - Saving checkpoint at 1 epochs
[ ] 0/6019, elapsed: 0s, ETA:<class 'mmcv.parallel.data_container.DataContainer'>
<class 'mmcv.parallel.data_container.DataContainer'>
Traceback (most recent call last):
File "./tools/train.py", line 283, in
Traceback (most recent call last):
File "./tools/train.py", line 283, in
main()
File "./tools/train.py", line 272, in main
main()
File "./tools/train.py", line 272, in main
train_detector(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
train_detector(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)runner.run(data_loaders, cfg.workflow)

File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 54, in train
epoch_runner(data_loaders[i], **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 54, in train
self.call_hook('after_train_epoch')
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 308, in call_hook
self.call_hook('after_train_epoch')
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 308, in call_hook
getattr(hook, fn_name)(self)
getattr(hook, fn_name)(self) File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/core/evaluation/eval_hooks.py", line 272, in after_train_epoch

File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/core/evaluation/eval_hooks.py", line 272, in after_train_epoch
results = multi_gpu_test(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/test.py", line 97, in multi_gpu_test
results = multi_gpu_test(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/test.py", line 97, in multi_gpu_test
result = model(return_loss=False, rescale=True, **data)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
result = model(return_loss=False, rescale=True, **data)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 799, in forward
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 799, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
output = self.module(*inputs[0], **kwargs[0])
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 60, in forward
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 60, in forward
return self.forward_test(**kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 41, in forward_test
return self.forward_test(**kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 41, in forward_test
return self.simple_test(points[0], img_metas[0], img[0], **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 594, in simple_test
return self.simple_test(points[0], img_metas[0], img[0], **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 594, in simple_test
img_feats, pts_feats = self.extract_feat(
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 457, in extract_feat
img_feats, pts_feats = self.extract_feat(
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 457, in extract_feat
img_feats = self.extract_img_feat(img, img_metas)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 150, in extract_img_feat
img_feats = self.extract_img_feat(img, img_metas)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 150, in extract_img_feat
input_shape = img.shape[-2:]
input_shape = img.shape[-2:]
AttributeErrorAttributeError: : 'DataContainer' object has no attribute 'shape''DataContainer' object has no attribute 'shape'

When the training epoch[1] is finished, the error AttributeErrorAttributeError: : 'DataContainer' object has no attribute 'shape' 'DataContainer' object has no attribute 'shape'. How to solve this problem?

我弄了一个融合模型，我在验证集上跑的效果是mAP0.696，nds是0.731，但是提交的效果很差，mAp是0.64，一般是什么原因导致的，提交的格式有改变吗

Checklist

1. I have searched related issues but cannot get the expected help.
2. The bug has not been fixed in the latest version.

Describe the bug

AttributeError: module 'pycocotools' has no attribute '__version__'

Hello, I followed your environment setup and installed mmcv-full=1.3.0, mmdet=2.10.0, mmdet3d=0.10.0, mmpycocotools=12.0.3 and pycocotools=2.0.1. During the verification process, I tried to run demo.py to validate the successful installation of mmdet3d. However, I encountered an error that says "AttributeError: module 'pycocotools' has no attribute 'version'". I checked the faq.md , you mentioned that uninstalling mmpycocotools and pycocotools, then reinstalling mmpycocotools could resolve the issue. I tried this solution, but it didn't work. If I only install mmpycocotools, a new issue appeared.

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject

So I found the solution in the mmdetection3d documentation. The solution proposed there is similar to your suggestion, but it includes an additional step of installing pycocotools==2.0.1. However, the bug ‘’AttributeError: module 'pycocotools' has no attribute 'version' ‘’ recurrence when I installed pycocotools.

Reproduction

1. What command or script did you run?

python demo/pcd_demo.py demo/kitti_000008.bin configs/second/hv_second_secfpn_6x8_80e_kitti-3d-car.py checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-car_20200620_230238-393f000c.pth

2. Did you make any modifications on the code or config? Did you understand what you have modified?
   I didn't make any modifications.
3. What dataset did you use?
   demo.py

Environment

1. Please run python mmdet3d/utils/collect_env.py to collect necessary environment infomation and paste it here.

sys.platform: linux
Python: 3.8.16 (default, Mar  2 2023, 03:21:46) [GCC 11.2.0]
CUDA available: True
GPU 0: NVIDIA GeForce RTX 2080 Ti
CUDA_HOME: /usr/local/cuda
NVCC: Build cuda_11.1.TC455_06.29190527_0
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.9.0+cu111
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.1.2 (Git Hash 98be7e8afa711dc9b66c8ff3504129cb82013cdb)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.0.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

TorchVision: 0.10.0+cu111
OpenCV: 4.7.0
MMCV: 1.3.0
MMCV Compiler: GCC 7.5
MMCV CUDA Compiler: 11.1
MMDetection: 2.10.0
MMDetection3D: 0.10.0+1b39a48

3. You may add addition that may be helpful for locating the problem, such as
   • How you installed PyTorch [e.g., pip, conda, source]

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

Error traceback
If applicable, paste the error trackback here.

Traceback (most recent call last):
  File "demo/pcd_demo.py", line 3, in <module>
    from mmdet3d.apis import inference_detector, init_detector, show_result_meshlab
  File "/root/autodl-tmp/project/mmdetection3d/mmdet3d/apis/__init__.py", line 1, in <module>
    from .inference import inference_detector, init_detector, show_result_meshlab
  File "/root/autodl-tmp/project/mmdetection3d/mmdet3d/apis/inference.py", line 10, in <module>
    from mmdet3d.datasets.pipelines import Compose
  File "/root/autodl-tmp/project/mmdetection3d/mmdet3d/datasets/__init__.py", line 1, in <module>
    from mmdet.datasets.builder import build_dataloader
  File "/root/autodl-tmp/project/mmdetection/mmdet/datasets/__init__.py", line 2, in <module>
    from .cityscapes import CityscapesDataset
  File "/root/autodl-tmp/project/mmdetection/mmdet/datasets/cityscapes.py", line 16, in <module>
    from .coco import CocoDataset
  File "/root/autodl-tmp/project/mmdetection/mmdet/datasets/coco.py", line 21, in <module>
    assert pycocotools.__version__ >= '12.0.2'
AttributeError: module 'pycocotools' has no attribute '__version__'

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Hi, I try to have a inference by fusion_voxel0075_R50.pth(from Baidu cloud storage) and transfusion_nusc_voxel_L.py(base line)
When I run "python tools/test.py configs/transfusion_nusc_voxel_L.py checkpoints/fusion_voxel0075_R50.pth --eval bbox", the following error occur:

What I need to do for implement of this inference

Thanks for your great works! I notice that TTA can greatly improve the performance in many works. I want to know how to ensemble results from different configs. I tried to adjust the weighted box fusion into 3D format, but I cannot get better results. Can you provide some suggestions? Thanks!

Hi!

Due to the use of the offline virtual points, I wonder whether this code is applicable to dataset: nuscenes-mini?

Hope for your response!
Best

I encounterd this error when training the second model. Is anyone met the same error? If so, please teach me how to solve this. Thanks a lot.

Main library version :
torch 1.10.0+cu111
spconv-cu111 2.1.25
mmcv-full 1.4.0
mmdet 2.11.0
mmdet3d 0.11.0

Error:
[Exception|implicit_gemm]feat=torch.Size([290557, 96]),w=torch.Size([128, 3, 3, 3, 96]),pair=torch.Size([27, 131609]),act=131609,issubm=False,istrain=True
SPCONV_DEBUG_SAVE_PATH not found, you can specify SPCONV_DEBUG_SAVE_PATH as debug data save path to save debug data which can be attached in a issue.
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
Traceback (most recent call last):
File "./tools/train.py", line 287, in
main()
File "./tools/train.py", line 283, in main
meta=meta)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
epoch_runner(data_loaders[i], **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
**kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/parallel/distributed.py", line 52, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_func
return old_func(*args, **kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 530, in forward_train
points, img=img, img_metas=img_metas)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 458, in extract_feat
pts_feats = self.extract_pts_feat(points, img_feats, img_metas)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 438, in extract_pts_feat
self.radius_list, self.max_cluster_samples_list, self.dist_thresh_list)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/middle_encoders/sparse_multimodal_encoder_painting.py", line 456, in forward
voxel_stage_out_ds = getattr(self.downscale_blocks, stage_name)(voxel_stage_out)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/modules.py", line 137, in forward
input = module(input)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/conv.py", line 447, in forward
input._timer, self.fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/cuda/amp/autocast_mode.py", line 94, in decorate_fwd
return fwd(*args, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/functional.py", line 200, in forward
raise e
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/functional.py", line 191, in forward
fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/ops.py", line 1118, in implicit_gemm
fp32_accum=fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/algo.py", line 620, in tune_and_cache
output = output.clone()
RuntimeError: TensorStorage /io/include/tensorview/tensor.h 168
cuda failed with error 2 out of memory. use CUDA_LAUNCH_BLOCKING=1 to get correct traceback.

Traceback (most recent call last):
File "./tools/train.py", line 287, in
main()
File "./tools/train.py", line 283, in main
meta=meta)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
epoch_runner(data_loaders[i], **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
**kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/parallel/distributed.py", line 52, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_func
return old_func(*args, **kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 530, in forward_train
points, img=img, img_metas=img_metas)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 458, in extract_feat
pts_feats = self.extract_pts_feat(points, img_feats, img_metas)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 438, in extract_pts_feat
self.radius_list, self.max_cluster_samples_list, self.dist_thresh_list)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/middle_encoders/sparse_multimodal_encoder_painting.py", line 456, in forward
voxel_stage_out_ds = getattr(self.downscale_blocks, stage_name)(voxel_stage_out)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/modules.py", line 137, in forward
input = module(input)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/conv.py", line 447, in forward
input._timer, self.fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/cuda/amp/autocast_mode.py", line 94, in decorate_fwd
return fwd(*args, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/functional.py", line 200, in forward
raise e
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/functional.py", line 191, in forward
fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/ops.py", line 1118, in implicit_gemm
fp32_accum=fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/algo.py", line 620, in tune_and_cache
output = output.clone()
RuntimeError: TensorStorage /io/include/tensorview/tensor.h 168
cuda failed with error 2 out of memory. use CUDA_LAUNCH_BLOCKING=1 to get correct traceback.

fg_info = img_metas[sample_idx]['foreground2D_info']
fg_pxl = fg_info['fg_pixels'][view_idx]
fg_depth = torch.from_numpy(fg_pxl[:,2]).to(device) in line 207 in MSMDFusionDetector
and
fg_real_pixels = img_metas[i]['foreground2D_info']['fg_real_pixels']
depth = fg_real_pxl[:,2]

code above shows that you have use two depth information in the generation of foreground2D, could you tell me the deffrence between them ?

Is there a schedule for the code release?

Hello, thanks for your great work.
I found that your TTA (scaling/double flip) is different from other methods (rotation/double flip) on nuscenes leaderboard, could you please tell the reason?

Hello! I want to ask about the differences between TTA and the base version. And why can the TTA version improve such a large margin?

In the paper, you cat the pcd and img_feat into batch_fg_pcd_cams, and then voxelize batch_fg_pcd_cams. But voxelization will loss grad, so how does the parameter in the img_backbone be optimized?

Hope for your response!
Best

thanks for your great job!
I got some error when I untar data package FOREGROUND_6NN_WITH_DPETHS_sweeps.tar.gz.
Here is my command,
tar xvf FOREGROUND_6NN_WITH_DPETHS_sweeps.tar.gz

After minutes, there is something wrong happened. Could you repack data set?

gzip: stdin: invalid compressed data--crc error
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Hi, can you provide me with the training log files for the second stage of transfusion_nusc_voxel_LC and MSMDFusion_nusc_voxel_LC for my reference? Thank you!

#38 (comment)

Hi!

Do you have this .json file. I don't have enough GPU to generete it myself, I would really appreciate it if you could provide if for me :)

Hi,

Great work! I am trying to download the samples and sweeps. But I notice they are on baidu pan. The company I'm working with doesn't allow third part account logins (like baidu account) or using a usb on their devices. Do you think if you can upload them to google drive or microsoft onedrive? Sorry for the inconvenience

I note that you release the result of the model MSMDFusion (mAP 70.8, NSD 73.2) in Table 1 of your older version paper, but in the updated paper, you release the result of the model MSMDFusion-base (mAP 71.5, NSD 74.0) in Table 1. So what are the differences between these two models? Thanks in advance.

Hello, thank you very much for this excellent work. I have greatly benefited from reading your article. However, I have encountered some issues when running the code.
Firstly, in the article, it mentioned that the recommended version of MMDetection3D is 0.11.0. Should I follow the installation instructions provided or export the Python path like(export PYTHONPATH=${PYTHONPATH}:/root/MSMDFusion-main)?
Secondly, when I ran the demo, I encountered the following problem.When i git clone mmdet3d and pip install -v -e , i get this bug
`Created wheel for networkx: filename=networkx-2.2-py2.py3-none-any.whl size=1526911 sha256=d76950b5f10f72129bc4e681e812ee47f321c7fda90c90e65ed5439da3efa28e
Stored in directory: /root/.cache/pip/wheels/49/fb/7f/02c31ca537b34e1073844b733832e4c3a94071d8edda2c0faa
Successfully built networkx
Installing collected packages: networkx, mmdet3d
Attempting uninstall: networkx
Found existing installation: networkx 2.6.3
Uninstalling networkx-2.6.3:
Removing file or directory /root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/networkx-2.6.3.dist-info/
Removing file or directory /root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/networkx/
Removing file or directory /root/anaconda3/envs/MSMDFusion/share/doc/networkx-2.6.3/
Successfully uninstalled networkx-2.6.3
Attempting uninstall: mmdet3d
Found existing installation: mmdet3d 0.11.0
Can't uninstall 'mmdet3d'. No files were found to uninstall.
Running setup.py develop for mmdet3d
Running command python setup.py develop
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
Compiling sparse_conv_ext without CUDA
Compiling iou3d_cuda without CUDA
Compiling voxel_layer without CUDA
Compiling roiaware_pool3d_ext without CUDA
Compiling ball_query_ext without CUDA
Compiling group_points_ext without CUDA
Compiling interpolate_ext without CUDA
Compiling furthest_point_sample_ext without CUDA
Compiling gather_points_ext without CUDA
running develop
running egg_info
writing mmdet3d.egg-info/PKG-INFO
writing dependency_links to mmdet3d.egg-info/dependency_links.txt
writing requirements to mmdet3d.egg-info/requires.txt
writing top-level names to mmdet3d.egg-info/top_level.txt
reading manifest file 'mmdet3d.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  easy_install.initialize_options(self)
/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
warning: no files found matching 'mmdet3d/ops/**/*.cpp'
warning: no files found matching 'mmdet3d/ops/**/*.cu'
warning: no files found matching 'mmdet3d/ops/**/*.h'
warning: no files found matching 'mmdet3d/ops/**/*.cc'
warning: no files found matching 'mmdet3d/VERSION'
adding license file 'LICENSE'
writing manifest file 'mmdet3d.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "<string>", line 36, in <module>
  File "<pip-setuptools-caller>", line 34, in <module>
  File "/root/MSMDFusion-main/mmdetection3d/setup.py", line 248, in <module>
    zip_safe=False)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/__init__.py", line 107, in setup
    return distutils.core.setup(**attrs)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/command/develop.py", line 34, in run
    self.install_for_development()
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/command/develop.py", line 111, in install_for_development
    self.run_command('build_ext')
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 84, in run
    _build_ext.run(self)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 434, in build_extensions
    self._check_cuda_version(compiler_name, compiler_version)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 808, in _check_cuda_version
    torch_cuda_version = packaging.version.parse(torch.version.cuda)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/version.py", line 52, in parse
    return Version(version)
  File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/version.py", line 196, in __init__
    match = self._regex.search(version)
TypeError: expected string or bytes-like object
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /root/anaconda3/envs/MSMDFusion/bin/python -c '
exec(compile('"'"''"'"''"'"'
# This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
#
# - It imports setuptools before invoking setup.py, to enable projects that directly
#   import from `distutils.core` to work with newer packaging standards.
# - It provides a clear error message when setuptools is not installed.
# - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
#   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
#     manifest_maker: standard file '"'"'-c'"'"' not found".
# - It generates a shim setup.py, for handling setup.cfg-only projects.
import os, sys, tokenize

try:
    import setuptools
except ImportError as error:
    print(
        "ERROR: Can not execute `setup.py` since setuptools is not available in "
        "the build environment.",
        file=sys.stderr,
    )
    sys.exit(1)

__file__ = %r
sys.argv[0] = __file__

if os.path.exists(__file__):
    filename = __file__
    with tokenize.open(__file__) as f:
        setup_py_code = f.read()
else:
    filename = "<auto-generated setuptools caller>"
    setup_py_code = "from setuptools import setup; setup()"

exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/root/MSMDFusion-main/mmdetection3d/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' develop --no-deps
cwd: /root/MSMDFusion-main/mmdetection3d/

ERROR: Can't roll back mmdet3d; was not uninstalled
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.`
meanwhile i export PAYTHON_PATH , I meet this issue like this
'
Traceback (most recent call last):
File "demo/pcd_demo.py", line 3, in
from mmdet3d.apis import inference_detector, init_detector, show_result_meshlab
File "/root/MSMDFusion-main/mmdet3d/apis/init.py", line 1, in
from .inference import (convert_SyncBN, inference_detector, init_detector,
File "/root/MSMDFusion-main/mmdet3d/apis/inference.py", line 8, in
from mmdet3d.core import Box3DMode, show_result
File "/root/MSMDFusion-main/mmdet3d/core/init.py", line 1, in
from .anchor import * # noqa: F401, F403
File "/root/MSMDFusion-main/mmdet3d/core/anchor/init.py", line 1, in
from mmdet.core.anchor import build_anchor_generator
File "/root/MSMDFusion-main/mmdetection-2.x/mmdet/core/init.py", line 3, in
from .bbox import * # noqa: F401, F403
File "/root/MSMDFusion-main/mmdetection-2.x/mmdet/core/bbox/init.py", line 8, in
from .samplers import (BaseSampler, CombinedSampler,
File "/root/MSMDFusion-main/mmdetection-2.x/mmdet/core/bbox/samplers/init.py", line 12, in
from .score_hlr_sampler import ScoreHLRSampler
File "/root/MSMDFusion-main/mmdetection-2.x/mmdet/core/bbox/samplers/score_hlr_sampler.py", line 3, in
from mmcv.ops import nms_match
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/mmcv/ops/init.py", line 2, in
from .assign_score_withk import assign_score_withk
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/mmcv/ops/assign_score_withk.py", line 6, in
'_ext', ['assign_score_withk_forward', 'assign_score_withk_backward'])
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/mmcv/_ext.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops19empty_memory_format4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEENS6_INS2_12MemoryFormatEEE
'

Hi, when I use the following command for evaluation:

Evaluation

sh . /tools/dist_test.sh . /configs/MSMDFusion_nusc_voxel_LC.py $ckpt_path$ 2 --eval bbox
The results obtained, are consistent with epoch_6 on the validation set. However, I noticed that the results you gave for the test set have a big improvement compared to the validation set.

When I modify the read test set directly and run it again, the following two errors are reported:
KeyError: Caught KeyError in DataLoader worker process 0.
KeyError: 'num_lidar_pts'

I tried searching for "How to evaluate on Nuscenes test set" but did not find a suitable solution, can you help me with this?

Report
./tools/dist_train.sh: 7: Bad substitution

when running
sh ./tools/dist_train.sh ./configs/transfusion_nusc_voxel_L.py 1

Hi，I just wonder the MSMDFusion-base‘s val performance， as I don’t see the result in lastest version paper，and does you keep using same input image resolution as transfusion？ thanks。

Congratulations on the successful acceptance of your paper by cvpr 2023, I would like to ask when the code corresponding to this article will be open source ah, very much hope to learn from the seniors!

Hello, I am confused about the GMA-Conv module in the MSMDFusion paper. Before this module, you divided the existing 3D space voxels into three categories: LiDAR-only (blue voxels), Camera-only (yellow voxels), and LiDAR and Camera combined (red voxels). From Figure 2, it appears that the yellow voxels are generated by the MDU module, but shouldn't the MDU module produce LiDAR and Camera combined voxels（red voxels）? In that case, where do the red voxels come from? If the camera does not have depth information provided by LiDAR, how does the pixel become a camera-only voxel? Perhaps my understanding is not accurate, so I would appreciate your guidance!

Hi!

Have you implemented the visiualization part? How can I implement this part?

Wish for your response!

In README.md, virtual points samples is mentioned. And the virtual points samples is saved in Baidu cloud storage.

**Step 2**: Download preprocessed [virtual points samples](https://pan.baidu.com/s/1IxqcGxNCFnmSZw7Dlu3Xig?pwd=9xcb)(extraction code: 9xcb) and [sweeps](https://pan.baidu.com/s/1qUeopFHCWrr35af2MGBSnw?pwd=2eg1)(extraction code: 2eg1) data. And put them under the above folder ```samples``` and ```sweeps```, respectively, and rename them as ```FOREGROUND_MIXED_6NN_WITH_DEPTH```.

What I want to know is, how to get virtual points samples by code?
Thanks!

Hi,

Thanks for your great work ! When I am reading your codes, I am confused about how you generate the virtual points by a specific seed, as described in your paper.

Looking forward to your answer.

Thanks.

I have already installed the mmcv-full, but there is still an import error. Can anyone help please?

pip list | grep mm
comm 0.1.4
mmcv 2.0.0rc4
mmcv-full 1.7.0
mmdet 3.0.0
mmengine 0.9.0

Traceback (most recent call last):
File "create_data.py", line 11, in
from tools.data_converter.create_gt_database import create_groundtruth_database
File "/home/bitcqic/MSMDFusion/MSMDFusion/tools/data_converter/create_gt_database.py", line 4, in
from mmcv import track_iter_progress
ImportError: cannot import name 'track_iter_progress' from 'mmcv'