sxjyjay / msmdfusion Goto Github PK
View Code? Open in Web Editor NEW[CVPR 2023] MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection
License: Apache License 2.0
[CVPR 2023] MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection
License: Apache License 2.0
Hello, thank you for your outstanding work. I am currently retrying your code.
The problem I encountered is when visualizing the demo and displaying this error:
ImportError: cannot import name 'init_ detector' from 'mmdet3d.apis' (/home/ps/yuji/MSMDFusion/mmdetection3d/mmdet3d/apis/init.py)
My command line is: Python demo/pcd_ demo.py demo/kitti_ 000008.bin configs/second/hv_ second_ secfpn_ 6x8_ 80e_ kitti-3d-car.py checkpoints/hv_ second_ secfpn_ 6x8_ 80e_ kitti-3d-car_ 20200620_ 230238-393f000c.pth --device 0 --out-dir /home/ps/yuji/MSMDFusion/outdir
According to the error report, it is due to an error in the file path of the import.
So I have two questions:
1.What is the difference between the mmdet3d folder and mmdetection3d?
2. If the visualized demo refers to the pcd_demo located in the mmdet3d folder, why is there an error that cannot be imported in mmdetection3d folder?
Looking forward to your reply. Thank you very much~
I can't understand Flc, where does it come from?
Thanks for your error report and we appreciate it a lot.
Checklist
Describe the bug
A clear and concise description of what the bug is.
Reproduction
sh ./tools/dist_test.sh ./configs/MSMDFusion_nusc_voxel_LC.py $ckpt_path$ 8 --eval bbox
A placeholder for the command.
Environment
python mmdet3d/utils/collect_env.py
to collect necessary environment infomation and paste it here.$PATH
, $LD_LIBRARY_PATH
, $PYTHONPATH
, etc.)conda install -c pytorch pytorch torchvision -y
pip install mmcv-full
pip install git+https://github.com/open-mmlab/mmdetection.git
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
pip install -v -e .
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
How much GPU memory is need to evaluate the model?
I´m running:python3 tools/test.py configs/MSMDFusion_nusc_voxel_LC.py checkpoint/MSMDFusion.pth --eval bbox
But get the error message: RuntimeError: CUDA out of memory. Tried to allocate 1.21 GiB (GPU 0; 7.77 GiB total capacity; 4.40 GiB already allocated; 1.21 GiB free; 5.05 GiB reserved in total by PyTorch)
Hello, thanks for your great work.
I found that your TTA (scaling/double flip) is different from other methods (rotation/double flip) on nuscenes leaderboard, could you please tell the reason?
Hi, I try to have a inference by fusion_voxel0075_R50.pth(from Baidu cloud storage) and transfusion_nusc_voxel_L.py(base line)
When I run "python tools/test.py configs/transfusion_nusc_voxel_L.py checkpoints/fusion_voxel0075_R50.pth --eval bbox", the following error occur:
What I need to do for implement of this inference
Hi!
Hope for yous reply!
Best
Hi!
Have you implemented the visiualization part? How can I implement this part?
Wish for your response!
I have already installed the mmcv-full, but there is still an import error. Can anyone help please?
pip list | grep mm
comm 0.1.4
mmcv 2.0.0rc4
mmcv-full 1.7.0
mmdet 3.0.0
mmengine 0.9.0
Traceback (most recent call last):
File "create_data.py", line 11, in
from tools.data_converter.create_gt_database import create_groundtruth_database
File "/home/bitcqic/MSMDFusion/MSMDFusion/tools/data_converter/create_gt_database.py", line 4, in
from mmcv import track_iter_progress
ImportError: cannot import name 'track_iter_progress' from 'mmcv'
I find the 3d_box_dim
is 9. I konw the meaning of the first 7 values. I wonder what the last 2 values represent.
Could you provide the information above? Such that i can tell whether if can run on my machine? Thanks a lot..
Hi,I just wonder the MSMDFusion-base‘s val performance, as I don’t see the result in lastest version paper,and does you keep using same input image resolution as transfusion? thanks。
In README.md, virtual points samples is mentioned. And the virtual points samples is saved in Baidu cloud storage.
**Step 2**: Download preprocessed [virtual points samples](https://pan.baidu.com/s/1IxqcGxNCFnmSZw7Dlu3Xig?pwd=9xcb)(extraction code: 9xcb) and [sweeps](https://pan.baidu.com/s/1qUeopFHCWrr35af2MGBSnw?pwd=2eg1)(extraction code: 2eg1) data. And put them under the above folder ```samples``` and ```sweeps```, respectively, and rename them as ```FOREGROUND_MIXED_6NN_WITH_DEPTH```.
What I want to know is, how to get virtual points samples by code?
Thanks!
Hi!
I noticed the nms_type
is none in config file. I wonder whether you have done the nms part during the 3Dbboxes generating part.
wish for your response
Describe the bug
Hi I was wondering if someone else had issues when building the docker, I get compatibility issues.
Reproduction
(base) henrik@henrik-XPS-15-9510:~/mmdetection/mmdetection3d/MSMDFusion/docker$ docker build -t msmd .
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub
---------------------------------------------------Now the docker file looks like this-------------------------------------------------------------------
ARG PYTORCH="1.6.0"
ARG CUDA="10.1"
ARG CUDNN="7"
FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel
ENV TORCH_CUDA_ARCH_LIST="6.0 6.1 7.0+PTX"
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"
ENV CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"
### To fix GPG key error when running apt-get update
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub
RUN apt-get update && apt-get install -y ffmpeg libsm6 libxext6 git ninja-build libglib2.0-0 libsm6 libxrender-dev libxext6 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Install MMCV
RUN pip install mmcv-full==1.3.0
RUN pip install mmdet
# Install MMDetection
RUN conda clean --all
RUN git clone https://github.com/open-mmlab/mmdetection3d.git /mmdetection3d
WORKDIR /mmdetection3d
ENV FORCE_CUDA="1"
RUN pip install -r requirements/build.txt
RUN pip install --no-cache-dir -e .
# uninstall pycocotools installed by nuscenes-devkit and reinstall mmpycocotools
RUN pip uninstall pycocotools --no-cache-dir -y
RUN pip install mmpycocotools --no-cache-dir --force --no-deps
I downloaded the nuscenes dataset.
python mmdet3d/utils/collect_env.py
to collect necessary environment infomation and paste it here:TorchVision: 0.15.0
OpenCV: 4.9.0
MMEngine: 0.10.3
MMDetection: 3.3.0
MMDetection3D: 1.4.0+fe25f7a
spconv2.0: False
3. You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as $PATH
, $LD_LIBRARY_PATH
, $PYTHONPATH
, etc.)
Error traceback
If applicable, paste the error trackback here.
base) henrik@henrik-XPS-15-9510:~/mmdetection/mmdetection3d/MSMDFusion/docker$ docker build -t msmd .
[+] Building 92.3s (14/16) docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.29kB 0.0s
=> [internal] load metadata for docker.io/pytorch/pytorch:1.6.0-cuda10.1-cudnn7-devel 0.9s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [ 1/13] FROM docker.io/pytorch/pytorch:1.6.0-cuda10.1-cudnn7-devel@sha256:ccebb46f954b1d32a4700aaeae0e24bd68653f92c6f276a608bf592b660b63d7 0.0s
=> CACHED [ 2/13] RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub 0.0s
=> CACHED [ 3/13] RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub 0.0s
=> CACHED [ 4/13] RUN apt-get update && apt-get install -y ffmpeg libsm6 libxext6 git ninja-build libglib2.0-0 libsm6 libxrender-dev libxext6 && apt-get clean && rm -rf /var/lib/apt/lists/ 0.0s
=> CACHED [ 5/13] RUN pip install mmcv-full==1.3.0 0.0s
=> CACHED [ 6/13] RUN pip install mmdet 0.0s
=> CACHED [ 7/13] RUN conda clean --all 0.0s
=> CACHED [ 8/13] RUN git clone https://github.com/open-mmlab/mmdetection3d.git /mmdetection3d 0.0s
=> CACHED [ 9/13] WORKDIR /mmdetection3d 0.0s
=> CACHED [10/13] RUN pip install -r requirements/build.txt 0.0s
=> ERROR [11/13] RUN pip install --no-cache-dir -e . 91.4s
------
> [11/13] RUN pip install --no-cache-dir -e .:
0.568 Obtaining file:///mmdetection3d
1.645 Collecting lyft_dataset_sdk
1.719 Downloading lyft_dataset_sdk-0.0.8-py2.py3-none-any.whl (32 kB)
1.794 Collecting networkx>=2.5
1.814 Downloading networkx-2.6.3-py3-none-any.whl (1.9 MB)
2.378 Collecting numba
2.395 Downloading numba-0.56.4-cp37-cp37m-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.5 MB)
2.891 Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from mmdet3d==1.4.0) (1.18.5)
2.931 Collecting nuscenes-devkit
2.949 Downloading nuscenes_devkit-1.1.11-py3-none-any.whl (313 kB)
4.186 Collecting open3d
3.327 Downloading open3d-0.13.0-cp37-cp37m-manylinux2014_x86_64.whl (300.6 MB)
44.41 Collecting plyfile
44.43 Downloading plyfile-0.9-py3-none-any.whl (30 kB)
44.56 Collecting scikit-image
44.58 Downloading scikit_image-0.19.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.5 MB)
45.93 Collecting tensorboard
45.95 Downloading tensorboard-2.11.2-py3-none-any.whl (6.0 MB)
46.82 Collecting trimesh
46.83 Downloading trimesh-4.1.3-py3-none-any.whl (690 kB)
46.98 Collecting cachetools>=3.1.0
46.99 Downloading cachetools-5.3.2-py3-none-any.whl (9.3 kB)
47.00 Requirement already satisfied: opencv-python>=3.4.2.17 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (4.9.0.80)
47.26 Collecting pandas
47.29 Downloading pandas-1.3.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
48.60 Collecting black
48.62 Downloading black-23.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
48.86 Collecting flake8
48.87 Downloading flake8-5.0.4-py2.py3-none-any.whl (61 kB)
48.97 Collecting pytest
48.99 Downloading pytest-7.4.4-py3-none-any.whl (325 kB)
49.12 Collecting plotly
49.14 Downloading plotly-5.18.0-py3-none-any.whl (15.6 MB)
51.57 Collecting pyquaternion>=0.9.5
51.59 Downloading pyquaternion-0.9.9-py3-none-any.whl (14 kB)
51.61 Requirement already satisfied: Pillow>=5.2.0 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (7.2.0)
51.62 Requirement already satisfied: matplotlib in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (3.5.3)
51.78 Collecting scikit-learn>=0.19.2
51.79 Downloading scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
54.80 Requirement already satisfied: Shapely>=1.6.4.post2 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (2.0.2)
54.84 Collecting fire
54.86 Downloading fire-0.5.0.tar.gz (88 kB)
55.05 Requirement already satisfied: scipy>=1.1.0 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (1.7.3)
55.05 Requirement already satisfied: tqdm>=4.25.0 in /opt/conda/lib/python3.7/site-packages (from lyft_dataset_sdk->mmdet3d==1.4.0) (4.46.0)
55.06 Requirement already satisfied: importlib-metadata; python_version < "3.9" in /opt/conda/lib/python3.7/site-packages (from numba->mmdet3d==1.4.0) (6.7.0)
55.18 Collecting llvmlite<0.40,>=0.39.0dev0
55.20 Downloading llvmlite-0.39.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.6 MB)
59.10 Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from numba->mmdet3d==1.4.0) (46.4.0.post20200518)
59.10 Requirement already satisfied: pycocotools>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from nuscenes-devkit->mmdet3d==1.4.0) (2.0.7)
59.17 Collecting descartes
59.19 Downloading descartes-1.1.0-py3-none-any.whl (5.8 kB)
59.26 Collecting ipywidgets>=7.6.0
59.27 Downloading ipywidgets-8.1.2-py3-none-any.whl (139 kB)
59.46 Collecting jupyter-packaging~=0.10
59.48 Downloading jupyter_packaging-0.12.3-py3-none-any.whl (15 kB)
59.53 Collecting pygments>=2.7.4
59.54 Downloading pygments-2.17.2-py3-none-any.whl (1.2 MB)
59.74 Collecting wheel>=0.36.0
59.75 Downloading wheel-0.42.0-py3-none-any.whl (65 kB)
59.85 Collecting pyyaml>=5.4.1
59.86 Downloading PyYAML-6.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (670 kB)
60.17 Collecting jupyterlab==3.*,>=3.0.0
60.19 Downloading jupyterlab-3.6.7-py3-none-any.whl (8.9 MB)
61.38 Requirement already satisfied: addict in /opt/conda/lib/python3.7/site-packages (from open3d->mmdet3d==1.4.0) (2.4.0)
61.44 Collecting imageio>=2.4.1
61.45 Downloading imageio-2.31.2-py3-none-any.whl (313 kB)
61.59 Collecting tifffile>=2019.7.26
61.60 Downloading tifffile-2021.11.2-py3-none-any.whl (178 kB)
61.71 Collecting PyWavelets>=1.1.1
61.74 Downloading PyWavelets-1.3.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.4 MB)
62.46 Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.7/site-packages (from scikit-image->mmdet3d==1.4.0) (23.2)
63.30 Collecting grpcio>=1.24.3
63.33 Downloading grpcio-1.60.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)
64.03 Collecting tensorboard-plugin-wit>=1.6.0
64.05 Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
64.19 Collecting absl-py>=0.4
64.20 Downloading absl_py-2.1.0-py3-none-any.whl (133 kB)
64.34 Collecting google-auth<3,>=1.6.3
64.35 Downloading google_auth-2.27.0-py2.py3-none-any.whl (186 kB)
64.43 Collecting markdown>=2.6.8
64.44 Downloading Markdown-3.4.4-py3-none-any.whl (94 kB)
64.49 Collecting tensorboard-data-server<0.7.0,>=0.6.0
64.50 Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
65.35 Collecting protobuf<4,>=3.9.2
65.37 Downloading protobuf-3.20.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
65.54 Collecting werkzeug>=1.0.1
65.57 Downloading Werkzeug-2.2.3-py3-none-any.whl (233 kB)
65.60 Requirement already satisfied: requests<3,>=2.21.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard->mmdet3d==1.4.0) (2.23.0)
65.65 Collecting google-auth-oauthlib<0.5,>=0.4.1
65.67 Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
65.68 Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from pandas->lyft_dataset_sdk->mmdet3d==1.4.0) (2.8.2)
65.68 Requirement already satisfied: pytz>=2017.3 in /opt/conda/lib/python3.7/site-packages (from pandas->lyft_dataset_sdk->mmdet3d==1.4.0) (2020.1)
65.76 Collecting typed-ast>=1.4.2; python_version < "3.8" and implementation_name == "cpython"
65.78 Downloading typed_ast-1.5.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (778 kB)
65.91 Requirement already satisfied: typing-extensions>=3.10.0.0; python_version < "3.10" in /opt/conda/lib/python3.7/site-packages (from black->lyft_dataset_sdk->mmdet3d==1.4.0) (4.7.1)
65.94 Collecting pathspec>=0.9.0
65.96 Downloading pathspec-0.11.2-py3-none-any.whl (29 kB)
65.99 Collecting mypy-extensions>=0.4.3
66.01 Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
66.05 Collecting click>=8.0.0
66.07 Downloading click-8.1.7-py3-none-any.whl (97 kB)
66.09 Requirement already satisfied: tomli>=1.1.0; python_version < "3.11" in /opt/conda/lib/python3.7/site-packages (from black->lyft_dataset_sdk->mmdet3d==1.4.0) (2.0.1)
66.11 Requirement already satisfied: platformdirs>=2 in /opt/conda/lib/python3.7/site-packages (from black->lyft_dataset_sdk->mmdet3d==1.4.0) (4.0.0)
66.13 Collecting pyflakes<2.6.0,>=2.5.0
66.15 Downloading pyflakes-2.5.0-py2.py3-none-any.whl (66 kB)
66.19 Collecting pycodestyle<2.10.0,>=2.9.0
66.20 Downloading pycodestyle-2.9.1-py2.py3-none-any.whl (41 kB)
66.24 Collecting mccabe<0.8.0,>=0.7.0
66.26 Downloading mccabe-0.7.0-py2.py3-none-any.whl (7.3 kB)
66.30 Collecting exceptiongroup>=1.0.0rc8; python_version < "3.11"
66.32 Downloading exceptiongroup-1.2.0-py3-none-any.whl (16 kB)
66.35 Collecting pluggy<2.0,>=0.12
66.36 Downloading pluggy-1.2.0-py3-none-any.whl (17 kB)
66.40 Collecting iniconfig
66.41 Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
66.45 Collecting tenacity>=6.2.0
66.46 Downloading tenacity-8.2.3-py3-none-any.whl (24 kB)
66.47 Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.7/site-packages (from matplotlib->lyft_dataset_sdk->mmdet3d==1.4.0) (4.38.0)
66.49 Requirement already satisfied: pyparsing>=2.2.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->lyft_dataset_sdk->mmdet3d==1.4.0) (3.1.1)
66.50 Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->lyft_dataset_sdk->mmdet3d==1.4.0) (1.4.5)
66.50 Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.7/site-packages (from matplotlib->lyft_dataset_sdk->mmdet3d==1.4.0) (0.11.0)
66.54 Collecting joblib>=0.11
66.56 Downloading joblib-1.3.2-py3-none-any.whl (302 kB)
66.63 Collecting threadpoolctl>=2.0.0
66.64 Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
66.65 Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from fire->lyft_dataset_sdk->mmdet3d==1.4.0) (1.14.0)
66.68 Collecting termcolor
66.70 Downloading termcolor-2.3.0-py3-none-any.whl (6.9 kB)
66.71 Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.9"->numba->mmdet3d==1.4.0) (3.15.0)
66.72 Requirement already satisfied: traitlets>=4.3.1 in /opt/conda/lib/python3.7/site-packages (from ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (4.3.3)
66.73 Requirement already satisfied: ipython>=6.1.0 in /opt/conda/lib/python3.7/site-packages (from ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (7.16.1)
66.76 Collecting comm>=0.1.3
66.77 Downloading comm-0.1.4-py3-none-any.whl (6.6 kB)
66.84 Collecting widgetsnbextension~=4.0.10
66.86 Downloading widgetsnbextension-4.0.10-py3-none-any.whl (2.3 MB)
67.20 Collecting jupyterlab-widgets~=3.0.10
67.22 Downloading jupyterlab_widgets-3.0.10-py3-none-any.whl (215 kB)
67.30 Collecting tomlkit
67.32 Downloading tomlkit-0.12.3-py3-none-any.whl (37 kB)
67.35 Collecting deprecation
67.37 Downloading deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
67.42 Collecting jupyter-core
67.43 Downloading jupyter_core-4.12.0-py3-none-any.whl (89 kB)
67.60 Collecting jupyter-ydoc~=0.2.4
67.61 Downloading jupyter_ydoc-0.2.5-py3-none-any.whl (6.2 kB)
67.68 Collecting jupyterlab-server~=2.19
67.70 Downloading jupyterlab_server-2.24.0-py3-none-any.whl (57 kB)
67.85 Collecting jupyter-server-ydoc~=0.8.0
67.87 Downloading jupyter_server_ydoc-0.8.0-py3-none-any.whl (11 kB)
67.97 Collecting jupyter-server<3,>=1.16.0
67.98 Downloading jupyter_server-1.24.0-py3-none-any.whl (347 kB)
68.13 Collecting notebook<7
68.14 Downloading notebook-6.5.6-py3-none-any.whl (529 kB)
68.41 Collecting nbclassic
68.42 Downloading nbclassic-1.0.0-py3-none-any.whl (10.0 MB)
69.72 Requirement already satisfied: jinja2>=2.1 in /opt/conda/lib/python3.7/site-packages (from jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (2.11.2)
69.82 Collecting tornado>=6.1.0
69.84 Downloading tornado-6.2-cp37-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (423 kB)
69.95 Collecting rsa<5,>=3.1.4
69.96 Downloading rsa-4.9-py3-none-any.whl (34 kB)
70.02 Collecting pyasn1-modules>=0.2.1
70.03 Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
70.19 Collecting MarkupSafe>=2.1.1
70.21 Downloading MarkupSafe-2.1.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
70.21 Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->mmdet3d==1.4.0) (1.25.8)
70.23 Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->mmdet3d==1.4.0) (2.9)
70.23 Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->mmdet3d==1.4.0) (2020.6.20)
70.23 Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard->mmdet3d==1.4.0) (3.0.4)
70.25 Collecting requests-oauthlib>=0.7.0
70.27 Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
70.29 Requirement already satisfied: ipython-genutils in /opt/conda/lib/python3.7/site-packages (from traitlets>=4.3.1->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.2.0)
70.29 Requirement already satisfied: decorator in /opt/conda/lib/python3.7/site-packages (from traitlets>=4.3.1->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (4.4.2)
70.29 Requirement already satisfied: pexpect; sys_platform != "win32" in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (4.8.0)
70.29 Requirement already satisfied: pickleshare in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.7.5)
70.29 Requirement already satisfied: backcall in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.2.0)
70.29 Requirement already satisfied: jedi>=0.10 in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.17.1)
70.30 Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (3.0.5)
70.50 Collecting y-py<0.7.0,>=0.6.0
70.52 Downloading y_py-0.6.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
70.81 Collecting jsonschema>=4.17.3
70.83 Downloading jsonschema-4.17.3-py3-none-any.whl (90 kB)
70.89 Collecting json5>=0.9.0
70.91 Downloading json5-0.9.14-py2.py3-none-any.whl (19 kB)
70.94 Collecting babel>=2.10
70.96 Downloading Babel-2.14.0-py3-none-any.whl (11.0 MB)
72.44 Collecting ypy-websocket<0.9.0,>=0.8.2
72.45 Downloading ypy_websocket-0.8.4-py3-none-any.whl (10 kB)
72.59 Collecting jupyter-server-fileid<1,>=0.6.0
72.60 Downloading jupyter_server_fileid-0.9.1-py3-none-any.whl (16 kB)
72.66 Collecting argon2-cffi
72.68 Downloading argon2_cffi-23.1.0-py3-none-any.whl (15 kB)
72.73 Collecting websocket-client
72.75 Downloading websocket_client-1.6.1-py3-none-any.whl (56 kB)
72.81 Collecting jupyter-client>=6.1.12
72.83 Downloading jupyter_client-7.4.9-py3-none-any.whl (133 kB)
72.91 Collecting terminado>=0.8.3
72.92 Downloading terminado-0.17.1-py3-none-any.whl (17 kB)
72.96 Collecting nbformat>=5.2.0
72.98 Downloading nbformat-5.8.0-py3-none-any.whl (77 kB)
73.38 Collecting pyzmq>=17
73.41 Downloading pyzmq-25.1.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)
73.61 Collecting Send2Trash
73.63 Downloading Send2Trash-1.8.2-py3-none-any.whl (18 kB)
73.68 Collecting prometheus-client
73.69 Downloading prometheus_client-0.17.1-py3-none-any.whl (60 kB)
73.76 Collecting nbconvert>=6.4.4
73.78 Downloading nbconvert-7.6.0-py3-none-any.whl (290 kB)
73.89 Collecting anyio<4,>=3.1.0
73.90 Downloading anyio-3.7.1-py3-none-any.whl (80 kB)
73.99 Collecting ipykernel
74.01 Downloading ipykernel-6.16.2-py3-none-any.whl (138 kB)
74.08 Collecting nest-asyncio>=1.5
74.09 Downloading nest_asyncio-1.6.0-py3-none-any.whl (5.2 kB)
74.11 Collecting notebook-shim>=0.2.3
74.13 Downloading notebook_shim-0.2.3-py3-none-any.whl (13 kB)
74.21 Collecting pyasn1>=0.1.3
74.22 Downloading pyasn1-0.5.1-py2.py3-none-any.whl (84 kB)
74.26 Collecting oauthlib>=3.0.0
74.28 Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
74.33 Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.7/site-packages (from pexpect; sys_platform != "win32"->ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.6.0)
74.33 Requirement already satisfied: parso<0.8.0,>=0.7.0 in /opt/conda/lib/python3.7/site-packages (from jedi>=0.10->ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.7.0)
74.33 Requirement already satisfied: wcwidth in /opt/conda/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=6.1.0->ipywidgets>=7.6.0->open3d->mmdet3d==1.4.0) (0.2.5)
74.38 Collecting pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0
74.39 Downloading pyrsistent-0.19.3-py3-none-any.whl (57 kB)
74.45 Collecting importlib-resources>=1.4.0; python_version < "3.9"
74.47 Downloading importlib_resources-5.12.0-py3-none-any.whl (36 kB)
74.51 Collecting pkgutil-resolve-name>=1.3.10; python_version < "3.9"
74.53 Downloading pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
74.56 Collecting attrs>=17.4.0
74.57 Downloading attrs-23.2.0-py3-none-any.whl (60 kB)
74.73 Collecting aiosqlite<1,>=0.17.0
74.74 Downloading aiosqlite-0.19.0-py3-none-any.whl (15 kB)
74.80 Collecting aiofiles<23,>=22.1.0
74.81 Downloading aiofiles-22.1.0-py3-none-any.whl (14 kB)
74.85 Collecting jupyter-events>=0.5.0
74.86 Downloading jupyter_events-0.6.3-py3-none-any.whl (18 kB)
74.90 Collecting argon2-cffi-bindings
74.91 Downloading argon2_cffi_bindings-21.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (86 kB)
74.94 Collecting entrypoints
74.96 Downloading entrypoints-0.4-py3-none-any.whl (5.3 kB)
74.99 Collecting fastjsonschema
75.00 Downloading fastjsonschema-2.19.1-py3-none-any.whl (23 kB)
75.05 Collecting nbclient>=0.5.0
75.06 Downloading nbclient-0.7.4-py3-none-any.whl (73 kB)
75.13 Collecting tinycss2
75.14 Downloading tinycss2-1.2.1-py3-none-any.whl (21 kB)
75.19 Collecting bleach!=5.0.0
75.21 Downloading bleach-6.0.0-py3-none-any.whl (162 kB)
75.27 Collecting mistune<4,>=2.0.3
75.28 Downloading mistune-3.0.2-py3-none-any.whl (47 kB)
75.31 Collecting defusedxml
75.33 Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
75.36 Collecting pandocfilters>=1.4.1
75.37 Downloading pandocfilters-1.5.1-py2.py3-none-any.whl (8.7 kB)
75.38 Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.7/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=1.16.0->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (4.9.1)
75.41 Collecting jupyterlab-pygments
75.43 Downloading jupyterlab_pygments-0.2.2-py2.py3-none-any.whl (21 kB)
75.46 Collecting sniffio>=1.1
75.47 Downloading sniffio-1.3.0-py3-none-any.whl (10 kB)
75.69 Collecting debugpy>=1.0
75.71 Downloading debugpy-1.7.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
76.07 Requirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from ipykernel->notebook<7->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (5.7.0)
76.10 Collecting matplotlib-inline>=0.1
76.12 Downloading matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
76.16 Collecting rfc3986-validator>=0.1.1
76.17 Downloading rfc3986_validator-0.1.1-py2.py3-none-any.whl (4.2 kB)
76.20 Collecting python-json-logger>=2.0.4
76.21 Downloading python_json_logger-2.0.7-py3-none-any.whl (8.1 kB)
76.24 Collecting rfc3339-validator
76.26 Downloading rfc3339_validator-0.1.4-py2.py3-none-any.whl (3.5 kB)
76.27 Requirement already satisfied: cffi>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=1.16.0->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (1.14.0)
76.29 Collecting webencodings>=0.4
76.31 Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
76.32 Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.7/site-packages (from beautifulsoup4->nbconvert>=6.4.4->jupyter-server<3,>=1.16.0->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (2.0.1)
76.32 Requirement already satisfied: pycparser in /opt/conda/lib/python3.7/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=1.16.0->jupyterlab==3.*,>=3.0.0->open3d->mmdet3d==1.4.0) (2.20)
76.32 Building wheels for collected packages: fire
76.32 Building wheel for fire (setup.py): started
76.48 Building wheel for fire (setup.py): finished with status 'done'
76.48 Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116936 sha256=e71e38260e683cd2f241967c10362ed41096b703538c8eacc1df9483d439ca26
76.48 Stored in directory: /tmp/pip-ephem-wheel-cache-3gf77sm_/wheels/20/97/e1/dd2c472bebcdcaa85fdc07d0f19020299f1c86773028860c53
76.48 Successfully built fire
77.06 ERROR: flake8 5.0.4 has requirement importlib-metadata<4.3,>=1.1.0; python_version < "3.8", but you'll have importlib-metadata 6.7.0 which is incompatible.
77.06 ERROR: nuscenes-devkit 1.1.11 has requirement numpy>=1.22.0, but you'll have numpy 1.18.5 which is incompatible.
77.06 ERROR: nuscenes-devkit 1.1.11 has requirement Shapely<2.0.0, but you'll have shapely 2.0.2 which is incompatible.
77.06 ERROR: jupyter-packaging 0.12.3 has requirement setuptools>=60.2.0, but you'll have setuptools 46.4.0.post20200518 which is incompatible.
77.06 ERROR: nbformat 5.8.0 has requirement traitlets>=5.1, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: nbclient 0.7.4 has requirement traitlets>=5.3, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: nbconvert 7.6.0 has requirement jinja2>=3.0, but you'll have jinja2 2.11.2 which is incompatible.
77.06 ERROR: nbconvert 7.6.0 has requirement traitlets>=5.1, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: jupyter-server 1.24.0 has requirement traitlets>=5.1, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: jupyterlab-server 2.24.0 has requirement jinja2>=3.0.3, but you'll have jinja2 2.11.2 which is incompatible.
77.06 ERROR: jupyterlab-server 2.24.0 has requirement requests>=2.28, but you'll have requests 2.23.0 which is incompatible.
77.06 ERROR: jupyter-events 0.6.3 has requirement traitlets>=5.3, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: ipykernel 6.16.2 has requirement ipython>=7.23.1, but you'll have ipython 7.16.1 which is incompatible.
77.06 ERROR: ipykernel 6.16.2 has requirement traitlets>=5.1.0, but you'll have traitlets 4.3.3 which is incompatible.
77.06 ERROR: notebook 6.5.6 has requirement pyzmq<25,>=17, but you'll have pyzmq 25.1.2 which is incompatible.
77.06 ERROR: open3d 0.13.0 has requirement pillow>=8.2.0, but you'll have pillow 7.2.0 which is incompatible.
77.06 ERROR: imageio 2.31.2 has requirement pillow>=8.3.2, but you'll have pillow 7.2.0 which is incompatible.
77.06 Installing collected packages: cachetools, pandas, typed-ast, pathspec, mypy-extensions, click, black, pyflakes, pycodestyle, mccabe, flake8, exceptiongroup, pluggy, iniconfig, pytest, tenacity, plotly, pyquaternion, joblib, threadpoolctl, scikit-learn, termcolor, fire, lyft-dataset-sdk, networkx, llvmlite, numba, descartes, nuscenes-devkit, comm, widgetsnbextension, jupyterlab-widgets, ipywidgets, tomlkit, wheel, deprecation, jupyter-packaging, pygments, pyyaml, jupyter-core, y-py, jupyter-ydoc, pyrsistent, importlib-resources, pkgutil-resolve-name, attrs, jsonschema, argon2-cffi-bindings, argon2-cffi, websocket-client, tornado, entrypoints, nest-asyncio, pyzmq, jupyter-client, terminado, fastjsonschema, nbformat, Send2Trash, prometheus-client, nbclient, MarkupSafe, webencodings, tinycss2, bleach, mistune, defusedxml, pandocfilters, jupyterlab-pygments, nbconvert, sniffio, anyio, jupyter-server, json5, babel, jupyterlab-server, aiosqlite, aiofiles, ypy-websocket, rfc3986-validator, python-json-logger, rfc3339-validator, jupyter-events, jupyter-server-fileid, jupyter-server-ydoc, debugpy, matplotlib-inline, ipykernel, notebook-shim, nbclassic, notebook, jupyterlab, open3d, plyfile, imageio, tifffile, PyWavelets, scikit-image, grpcio, tensorboard-plugin-wit, absl-py, pyasn1, rsa, pyasn1-modules, google-auth, markdown, tensorboard-data-server, protobuf, werkzeug, oauthlib, requests-oauthlib, google-auth-oauthlib, tensorboard, trimesh, mmdet3d
89.48 Attempting uninstall: wheel
89.48 Found existing installation: wheel 0.34.2
89.48 Uninstalling wheel-0.34.2:
89.83 Successfully uninstalled wheel-0.34.2
89.87 Attempting uninstall: pygments
89.87 Found existing installation: Pygments 2.6.1
89.91 Uninstalling Pygments-2.6.1:
90.01 Successfully uninstalled Pygments-2.6.1
90.37 Attempting uninstall: pyyaml
90.37 Found existing installation: PyYAML 5.3.1
90.91 ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
------
Dockerfile:29
--------------------
27 | ENV FORCE_CUDA="1"
28 | RUN pip install -r requirements/build.txt
29 | >>> RUN pip install --no-cache-dir -e .
30 |
31 | # uninstall pycocotools installed by nuscenes-devkit and reinstall mmpycocotools
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install --no-cache-dir -e ." did not complete successfully: exit code: 1
Any help would be appreciated.
Hi, Thanks for your open-source code, I'm trying to reappear your code.
Although I complete the preprocess of nuscenes, I failed in training stage due to the unknown version of python library.
Could you provide the current version of cuda, python, pytorch, torch-scatter, mmcv-full, mmdetection, spconv-2 in MSMDFusion?
Thanks a lot!
Hi!
Thanks for your great work!
When I try to train the 1-st model, Transfusion-L,I noticed that you metioned the copy-and-paste fade strategy. I wonder to know how I can achieve this mothod. I will appreciate it if you can give an example.
Hope for your response!
Best
Hi, when I use the following command for evaluation:
sh . /tools/dist_test.sh . /configs/MSMDFusion_nusc_voxel_LC.py
The results obtained, are consistent with epoch_6 on the validation set. However, I noticed that the results you gave for the test set have a big improvement compared to the validation set.
When I modify the read test set directly and run it again, the following two errors are reported:
KeyError: Caught KeyError in DataLoader worker process 0.
KeyError: 'num_lidar_pts'
I tried searching for "How to evaluate on Nuscenes test set" but did not find a suitable solution, can you help me with this?
Hi, can you provide me with the training log files for the second stage of transfusion_nusc_voxel_LC and MSMDFusion_nusc_voxel_LC for my reference? Thank you!
Hi, Thanks for your open-source code, I'm trying to reappear your code.
But I encountered some problems, I successfully installed the environment according to#Issues 1, and also processed the data. But when I run it, it shows_**ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject**_
I searched for related solutions, then improved the numpy version. But mmdet3d 0.11.0 requires numpy<=1.20.0. I tried to improve the numpy version and run it, but there was a problem with the dataset.
Original Traceback (most recent call last):
I think it is the reason for the numpy version. Could you please provide some advice? Thanks a lot!
ys.platform: linux
Python: 3.8.16 (default, Mar 2 2023, 03:21:46) [GCC 11.2.0]
CUDA available: True
GPU 0,1: NVIDIA GeForce RTX 3090
CUDA_HOME: /home/lxt/cuda-11.1
NVCC: Build cuda_11.1.TC455_06.29190527_0
GCC: gcc (Ubuntu 5.5.0-12ubuntu1) 5.5.0 20171010
PyTorch: 1.9.0+cu111
PyTorch compiling details: PyTorch built with:
TorchVision: 0.10.0+cu111
OpenCV: 4.7.0
MMCV: 1.3.18
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.1
MMDetection: 2.10.0
MMDetection3D: 0.11.0+
Hello! I want to ask about the differences between TTA and the base version. And why can the TTA version improve such a large margin?
How did you evaluate tracking, do you use CenterPoint? If so does not CenterPoint want the checkpoint files to be JSON format instead of pth, how did you manage that?
Im using this command:
(msmd) henrik@henrik-XPS-15-9510:~/CenterPoint$ python tools/nusc_tracking/pub_test.py --work_dir configs/nusc/voxelnet --checkpoint checkpoint/MSMDFusion.pth
======
Loading NuScenes tables for version v1.0-trainval...
23 category,
8 attribute,
4 visibility,
64386 instance,
12 sensor,
10200 calibrated_sensor,
2631083 ego_pose,
68 log,
850 scene,
34149 sample,
2631083 sample_data,
1166187 sample_annotation,
4 map,
Done loading in 21.414 seconds.
======
Reverse indexing ...
Done reverse indexing in 5.6 seconds.
======
Deploy OK
Use hungarian: False
Traceback (most recent call last):
File "tools/nusc_tracking/pub_test.py", line 190, in <module>
main()
File "tools/nusc_tracking/pub_test.py", line 84, in main
predictions=json.load(f)['results']
File "/home/henrik/miniconda3/envs/msmd/lib/python3.7/json/__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/home/henrik/miniconda3/envs/msmd/lib/python3.7/json/__init__.py", line 343, in loads
s = s.decode(detect_encoding(s), 'surrogatepass')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte
Should I convert MSMDFusion checkpoint files to JSON format or should i change in CenterPoint file pub_test to handle pth files?
我弄了一个融合模型,我在验证集上跑的效果是mAP0.696,nds是0.731,但是提交的效果很差,mAp是0.64,一般是什么原因导致的,提交的格式有改变吗
Report
./tools/dist_train.sh: 7: Bad substitution
when running
sh ./tools/dist_train.sh ./configs/transfusion_nusc_voxel_L.py 1
Thanks for your great works! I notice that TTA can greatly improve the performance in many works. I want to know how to ensemble results from different configs. I tried to adjust the weighted box fusion into 3D format, but I cannot get better results. Can you provide some suggestions? Thanks!
I note that you release the result of the model MSMDFusion (mAP 70.8, NSD 73.2) in Table 1 of your older version paper, but in the updated paper, you release the result of the model MSMDFusion-base (mAP 71.5, NSD 74.0) in Table 1. So what are the differences between these two models? Thanks in advance.
Hi,
Thanks for your great work ! When I am reading your codes, I am confused about how you generate the virtual points by a specific seed, as described in your paper.
Looking forward to your answer.
Thanks.
Hi!
I have 2 questions:
Hope for your response.
Best
2023-10-09 11:44:22,955 - mmdet - INFO - Exp name: MSMDFusion_nusc_voxel_LC.py
2023-10-09 11:44:22,956 - mmdet - INFO - Epoch [1][32000/32025] lr: 1.000e-04, eta: 3 days, 4:50:02, time: 1.717, data_time: 0.076, memory: 8875, loss_heatmap: 0.4555, layer_-1_loss_cls: 0.0766, layer_-1_loss_bbox: 0.4751, matched_ious: 0.5853, loss: 1.0072, grad_norm: 1.5121
2023-10-09 11:45:07,229 - mmdet - INFO - Saving checkpoint at 1 epochs
[ ] 0/6019, elapsed: 0s, ETA:<class 'mmcv.parallel.data_container.DataContainer'>
<class 'mmcv.parallel.data_container.DataContainer'>
Traceback (most recent call last):
File "./tools/train.py", line 283, in
Traceback (most recent call last):
File "./tools/train.py", line 283, in
main()
File "./tools/train.py", line 272, in main
main()
File "./tools/train.py", line 272, in main
train_detector(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
train_detector(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)runner.run(data_loaders, cfg.workflow)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 54, in train
epoch_runner(data_loaders[i], **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 54, in train
self.call_hook('after_train_epoch')
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 308, in call_hook
self.call_hook('after_train_epoch')
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 308, in call_hook
getattr(hook, fn_name)(self)
getattr(hook, fn_name)(self) File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/core/evaluation/eval_hooks.py", line 272, in after_train_epoch
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/core/evaluation/eval_hooks.py", line 272, in after_train_epoch
results = multi_gpu_test(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/test.py", line 97, in multi_gpu_test
results = multi_gpu_test(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/test.py", line 97, in multi_gpu_test
result = model(return_loss=False, rescale=True, **data)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
result = model(return_loss=False, rescale=True, **data)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 799, in forward
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 799, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
output = self.module(*inputs[0], **kwargs[0])
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 60, in forward
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 60, in forward
return self.forward_test(**kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 41, in forward_test
return self.forward_test(**kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 41, in forward_test
return self.simple_test(points[0], img_metas[0], img[0], **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 594, in simple_test
return self.simple_test(points[0], img_metas[0], img[0], **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 594, in simple_test
img_feats, pts_feats = self.extract_feat(
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 457, in extract_feat
img_feats, pts_feats = self.extract_feat(
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 457, in extract_feat
img_feats = self.extract_img_feat(img, img_metas)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 150, in extract_img_feat
img_feats = self.extract_img_feat(img, img_metas)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 150, in extract_img_feat
input_shape = img.shape[-2:]
input_shape = img.shape[-2:]
AttributeErrorAttributeError: : 'DataContainer' object has no attribute 'shape''DataContainer' object has no attribute 'shape'
When the training epoch[1] is finished, the error AttributeErrorAttributeError: : 'DataContainer' object has no attribute 'shape' 'DataContainer' object has no attribute 'shape'. How to solve this problem?
Hi,I notice that you use resnet50+fpn to generate image feature,but use centernet2 to generate virtual point,so I just wonder wether the inference speed 2.1HZ include the centernet2 inference time and virtual point generation time?if not,after add that time,how is the inference speed?
Congratulations on the successful acceptance of your paper by cvpr 2023, I would like to ask when the code corresponding to this article will be open source ah, very much hope to learn from the seniors!
Hello, when I download the fusion_voxel0075_R50.pth you provided, and run sh . /tools/dist_train.sh . /configs/MSMDFusion_nusc_voxel_LC.py 2 for the 2-nd stage training, the error is reported as follows, tried some solutions on the Internet still did not get a solution, I hope you can point out, thank you!
2023-09-14 10:43:15,801 - mmdet - INFO - Start running, host: xzluo@b5163d5d11c9, work_dir: /public/home/xzluo/zc/MSMDFusion-main/work_dirs/MSMDFusion_nusc_voxel_LC
2023-09-14 10:43:15,801 - mmdet - INFO - workflow: [('train', 1)], max: 6 epochs
Traceback (most recent call last):
File "./tools/train.py", line 283, in
main()
File "./tools/train.py", line 272, in main
train_detector(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter
outputs = self.model.train_step(data_batch, self.optimizer,
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/parallel/distributed.py", line 46, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 534, in forward_train
losses_pts = self.forward_pts_train(pts_feats, img_feats, gt_bboxes_3d,
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 574, in forward_pts_train
losses = self.pts_bbox_head.loss(*loss_inputs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 164, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/dense_heads/transfusion_head.py", line 1260, in loss
layer_loss_cls = self.loss_cls(layer_cls_score, layer_labels, layer_label_weights, avg_factor=max(num_pos, 1))
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/losses/focal_loss.py", line 170, in forward
loss_cls = self.loss_weight * calculate_loss_func(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/losses/focal_loss.py", line 85, in sigmoid_focal_loss
loss = _sigmoid_focal_loss(pred.contiguous(), target, gamma, alpha, None,
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/ops/focal_loss.py", line 54, in forward
ext_module.sigmoid_focal_loss_forward(
RuntimeError: SigmoidFocalLoss is not compiled with GPU support
Traceback (most recent call last):
File "./tools/train.py", line 283, in
main()
File "./tools/train.py", line 272, in main
train_detector(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter
outputs = self.model.train_step(data_batch, self.optimizer,
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/parallel/distributed.py", line 46, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 534, in forward_train
losses_pts = self.forward_pts_train(pts_feats, img_feats, gt_bboxes_3d,
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 574, in forward_pts_train
losses = self.pts_bbox_head.loss(*loss_inputs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 164, in new_func
return old_func(*args, **kwargs)
File "/public/home/xzluo/zc/MSMDFusion-main/mmdet3d/models/dense_heads/transfusion_head.py", line 1260, in loss
layer_loss_cls = self.loss_cls(layer_cls_score, layer_labels, layer_label_weights, avg_factor=max(num_pos, 1))
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/losses/focal_loss.py", line 170, in forward
loss_cls = self.loss_weight * calculate_loss_func(
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmdet/models/losses/focal_loss.py", line 85, in sigmoid_focal_loss
loss = _sigmoid_focal_loss(pred.contiguous(), target, gamma, alpha, None,
File "/public/home/xzluo/anaconda3/envs/zc/lib/python3.8/site-packages/mmcv/ops/focal_loss.py", line 54, in forward
ext_module.sigmoid_focal_loss_forward(
RuntimeError: SigmoidFocalLoss is not compiled with GPU support
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 29983) of binary: /public/home/xzluo/anaconda3/envs/zc/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 3/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
I encounterd this error when training the second model. Is anyone met the same error? If so, please teach me how to solve this. Thanks a lot.
Main library version :
torch 1.10.0+cu111
spconv-cu111 2.1.25
mmcv-full 1.4.0
mmdet 2.11.0
mmdet3d 0.11.0
Error:
[Exception|implicit_gemm]feat=torch.Size([290557, 96]),w=torch.Size([128, 3, 3, 3, 96]),pair=torch.Size([27, 131609]),act=131609,issubm=False,istrain=True
SPCONV_DEBUG_SAVE_PATH not found, you can specify SPCONV_DEBUG_SAVE_PATH as debug data save path to save debug data which can be attached in a issue.
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
[WARNING]your gpu arch (8, 9) isn't compiled in prebuilt, may cause invalid device function. available: {(7, 0), (6, 1), (8, 0), (6, 0), (7, 5), (8, 6), (5, 2)}
Traceback (most recent call last):
File "./tools/train.py", line 287, in
main()
File "./tools/train.py", line 283, in main
meta=meta)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
epoch_runner(data_loaders[i], **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
**kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/parallel/distributed.py", line 52, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_func
return old_func(*args, **kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 530, in forward_train
points, img=img, img_metas=img_metas)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 458, in extract_feat
pts_feats = self.extract_pts_feat(points, img_feats, img_metas)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 438, in extract_pts_feat
self.radius_list, self.max_cluster_samples_list, self.dist_thresh_list)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/middle_encoders/sparse_multimodal_encoder_painting.py", line 456, in forward
voxel_stage_out_ds = getattr(self.downscale_blocks, stage_name)(voxel_stage_out)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/modules.py", line 137, in forward
input = module(input)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/conv.py", line 447, in forward
input._timer, self.fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/cuda/amp/autocast_mode.py", line 94, in decorate_fwd
return fwd(*args, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/functional.py", line 200, in forward
raise e
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/functional.py", line 191, in forward
fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/ops.py", line 1118, in implicit_gemm
fp32_accum=fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/algo.py", line 620, in tune_and_cache
output = output.clone()
RuntimeError: TensorStorage /io/include/tensorview/tensor.h 168
cuda failed with error 2 out of memory. use CUDA_LAUNCH_BLOCKING=1 to get correct traceback.
Traceback (most recent call last):
File "./tools/train.py", line 287, in
main()
File "./tools/train.py", line 283, in main
meta=meta)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
epoch_runner(data_loaders[i], **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
**kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/parallel/distributed.py", line 52, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_func
return old_func(*args, **kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 530, in forward_train
points, img=img, img_metas=img_metas)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 458, in extract_feat
pts_feats = self.extract_pts_feat(points, img_feats, img_metas)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/detectors/MSMDFusion.py", line 438, in extract_pts_feat
self.radius_list, self.max_cluster_samples_list, self.dist_thresh_list)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/MSMDFusion-main/mmdet3d/models/middle_encoders/sparse_multimodal_encoder_painting.py", line 456, in forward
voxel_stage_out_ds = getattr(self.downscale_blocks, stage_name)(voxel_stage_out)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/modules.py", line 137, in forward
input = module(input)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/conv.py", line 447, in forward
input._timer, self.fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/torch/cuda/amp/autocast_mode.py", line 94, in decorate_fwd
return fwd(*args, **kwargs)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/functional.py", line 200, in forward
raise e
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/functional.py", line 191, in forward
fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/pytorch/ops.py", line 1118, in implicit_gemm
fp32_accum=fp32_accum)
File "/root/autodl-tmp/envs/openMmlab/lib/python3.7/site-packages/spconv/algo.py", line 620, in tune_and_cache
output = output.clone()
RuntimeError: TensorStorage /io/include/tensorview/tensor.h 168
cuda failed with error 2 out of memory. use CUDA_LAUNCH_BLOCKING=1 to get correct traceback.
Checklist
Describe the bug
AttributeError: module 'pycocotools' has no attribute '__version__'
Hello, I followed your environment setup and installed mmcv-full=1.3.0, mmdet=2.10.0, mmdet3d=0.10.0, mmpycocotools=12.0.3 and pycocotools=2.0.1. During the verification process, I tried to run demo.py to validate the successful installation of mmdet3d. However, I encountered an error that says "AttributeError: module 'pycocotools' has no attribute 'version'". I checked the faq.md , you mentioned that uninstalling mmpycocotools and pycocotools, then reinstalling mmpycocotools could resolve the issue. I tried this solution, but it didn't work. If I only install mmpycocotools, a new issue appeared.
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject
So I found the solution in the mmdetection3d documentation. The solution proposed there is similar to your suggestion, but it includes an additional step of installing pycocotools==2.0.1. However, the bug ‘’AttributeError: module 'pycocotools' has no attribute 'version' ‘’ recurrence when I installed pycocotools.
Reproduction
python demo/pcd_demo.py demo/kitti_000008.bin configs/second/hv_second_secfpn_6x8_80e_kitti-3d-car.py checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-car_20200620_230238-393f000c.pth
Environment
python mmdet3d/utils/collect_env.py
to collect necessary environment infomation and paste it here.sys.platform: linux
Python: 3.8.16 (default, Mar 2 2023, 03:21:46) [GCC 11.2.0]
CUDA available: True
GPU 0: NVIDIA GeForce RTX 2080 Ti
CUDA_HOME: /usr/local/cuda
NVCC: Build cuda_11.1.TC455_06.29190527_0
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.9.0+cu111
PyTorch compiling details: PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.1.2 (Git Hash 98be7e8afa711dc9b66c8ff3504129cb82013cdb)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.1
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
- CuDNN 8.0.5
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
TorchVision: 0.10.0+cu111
OpenCV: 4.7.0
MMCV: 1.3.0
MMCV Compiler: GCC 7.5
MMCV CUDA Compiler: 11.1
MMDetection: 2.10.0
MMDetection3D: 0.10.0+1b39a48
pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
Error traceback
If applicable, paste the error trackback here.
Traceback (most recent call last):
File "demo/pcd_demo.py", line 3, in <module>
from mmdet3d.apis import inference_detector, init_detector, show_result_meshlab
File "/root/autodl-tmp/project/mmdetection3d/mmdet3d/apis/__init__.py", line 1, in <module>
from .inference import inference_detector, init_detector, show_result_meshlab
File "/root/autodl-tmp/project/mmdetection3d/mmdet3d/apis/inference.py", line 10, in <module>
from mmdet3d.datasets.pipelines import Compose
File "/root/autodl-tmp/project/mmdetection3d/mmdet3d/datasets/__init__.py", line 1, in <module>
from mmdet.datasets.builder import build_dataloader
File "/root/autodl-tmp/project/mmdetection/mmdet/datasets/__init__.py", line 2, in <module>
from .cityscapes import CityscapesDataset
File "/root/autodl-tmp/project/mmdetection/mmdet/datasets/cityscapes.py", line 16, in <module>
from .coco import CocoDataset
File "/root/autodl-tmp/project/mmdetection/mmdet/datasets/coco.py", line 21, in <module>
assert pycocotools.__version__ >= '12.0.2'
AttributeError: module 'pycocotools' has no attribute '__version__'
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
thanks for your great job!
I got some error when I untar data package FOREGROUND_6NN_WITH_DPETHS_sweeps.tar.gz.
Here is my command,
tar xvf FOREGROUND_6NN_WITH_DPETHS_sweeps.tar.gz
After minutes, there is something wrong happened. Could you repack data set?
gzip: stdin: invalid compressed data--crc error
tar: Child returned status 1
tar: Error is not recoverable: exiting now
fg_info = img_metas[sample_idx]['foreground2D_info']
fg_pxl = fg_info['fg_pixels'][view_idx]
fg_depth = torch.from_numpy(fg_pxl[:,2]).to(device) in line 207 in MSMDFusionDetector
and
fg_real_pixels = img_metas[i]['foreground2D_info']['fg_real_pixels']
depth = fg_real_pxl[:,2]
code above shows that you have use two depth information in the generation of foreground2D, could you tell me the deffrence between them ?
Hello, I am confused about the GMA-Conv module in the MSMDFusion paper. Before this module, you divided the existing 3D space voxels into three categories: LiDAR-only (blue voxels), Camera-only (yellow voxels), and LiDAR and Camera combined (red voxels). From Figure 2, it appears that the yellow voxels are generated by the MDU module, but shouldn't the MDU module produce LiDAR and Camera combined voxels(red voxels)? In that case, where do the red voxels come from? If the camera does not have depth information provided by LiDAR, how does the pixel become a camera-only voxel? Perhaps my understanding is not accurate, so I would appreciate your guidance!
Hi,
Great work! I am trying to download the samples and sweeps. But I notice they are on baidu pan. The company I'm working with doesn't allow third part account logins (like baidu account) or using a usb on their devices. Do you think if you can upload them to google drive or microsoft onedrive? Sorry for the inconvenience
Is there a schedule for the code release?
Hello, thank you very much for this excellent work. I have greatly benefited from reading your article. However, I have encountered some issues when running the code.
Firstly, in the article, it mentioned that the recommended version of MMDetection3D is 0.11.0. Should I follow the installation instructions provided or export the Python path like(export PYTHONPATH=${PYTHONPATH}:/root/MSMDFusion-main)?
Secondly, when I ran the demo, I encountered the following problem.When i git clone mmdet3d and pip install -v -e , i get this bug
`Created wheel for networkx: filename=networkx-2.2-py2.py3-none-any.whl size=1526911 sha256=d76950b5f10f72129bc4e681e812ee47f321c7fda90c90e65ed5439da3efa28e
Stored in directory: /root/.cache/pip/wheels/49/fb/7f/02c31ca537b34e1073844b733832e4c3a94071d8edda2c0faa
Successfully built networkx
Installing collected packages: networkx, mmdet3d
Attempting uninstall: networkx
Found existing installation: networkx 2.6.3
Uninstalling networkx-2.6.3:
Removing file or directory /root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/networkx-2.6.3.dist-info/
Removing file or directory /root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/networkx/
Removing file or directory /root/anaconda3/envs/MSMDFusion/share/doc/networkx-2.6.3/
Successfully uninstalled networkx-2.6.3
Attempting uninstall: mmdet3d
Found existing installation: mmdet3d 0.11.0
Can't uninstall 'mmdet3d'. No files were found to uninstall.
Running setup.py develop for mmdet3d
Running command python setup.py develop
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
Compiling sparse_conv_ext without CUDA
Compiling iou3d_cuda without CUDA
Compiling voxel_layer without CUDA
Compiling roiaware_pool3d_ext without CUDA
Compiling ball_query_ext without CUDA
Compiling group_points_ext without CUDA
Compiling interpolate_ext without CUDA
Compiling furthest_point_sample_ext without CUDA
Compiling gather_points_ext without CUDA
running develop
running egg_info
writing mmdet3d.egg-info/PKG-INFO
writing dependency_links to mmdet3d.egg-info/dependency_links.txt
writing requirements to mmdet3d.egg-info/requires.txt
writing top-level names to mmdet3d.egg-info/top_level.txt
reading manifest file 'mmdet3d.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` and ``easy_install``.
Instead, use pypa/build, pypa/installer or other
standards-based tools.
See https://github.com/pypa/setuptools/issues/917 for details.
********************************************************************************
!!
easy_install.initialize_options(self)
/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer or other
standards-based tools.
See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************
!!
self.initialize_options()
warning: no files found matching 'mmdet3d/ops/**/*.cpp'
warning: no files found matching 'mmdet3d/ops/**/*.cu'
warning: no files found matching 'mmdet3d/ops/**/*.h'
warning: no files found matching 'mmdet3d/ops/**/*.cc'
warning: no files found matching 'mmdet3d/VERSION'
adding license file 'LICENSE'
writing manifest file 'mmdet3d.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
File "<string>", line 36, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/root/MSMDFusion-main/mmdetection3d/setup.py", line 248, in <module>
zip_safe=False)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/command/develop.py", line 111, in install_for_development
self.run_command('build_ext')
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
self.build_extensions()
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 434, in build_extensions
self._check_cuda_version(compiler_name, compiler_version)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 808, in _check_cuda_version
torch_cuda_version = packaging.version.parse(torch.version.cuda)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/version.py", line 52, in parse
return Version(version)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/version.py", line 196, in __init__
match = self._regex.search(version)
TypeError: expected string or bytes-like object
error: subprocess-exited-with-error
× python setup.py develop did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /root/anaconda3/envs/MSMDFusion/bin/python -c '
exec(compile('"'"''"'"''"'"'
# This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
#
# - It imports setuptools before invoking setup.py, to enable projects that directly
# import from `distutils.core` to work with newer packaging standards.
# - It provides a clear error message when setuptools is not installed.
# - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
# setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
# manifest_maker: standard file '"'"'-c'"'"' not found".
# - It generates a shim setup.py, for handling setup.cfg-only projects.
import os, sys, tokenize
try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute `setup.py` since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)
__file__ = %r
sys.argv[0] = __file__
if os.path.exists(__file__):
filename = __file__
with tokenize.open(__file__) as f:
setup_py_code = f.read()
else:
filename = "<auto-generated setuptools caller>"
setup_py_code = "from setuptools import setup; setup()"
exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/root/MSMDFusion-main/mmdetection3d/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' develop --no-deps
cwd: /root/MSMDFusion-main/mmdetection3d/
ERROR: Can't roll back mmdet3d; was not uninstalled
error: subprocess-exited-with-error
× python setup.py develop did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.`
meanwhile i export PAYTHON_PATH , I meet this issue like this
'
Traceback (most recent call last):
File "demo/pcd_demo.py", line 3, in
from mmdet3d.apis import inference_detector, init_detector, show_result_meshlab
File "/root/MSMDFusion-main/mmdet3d/apis/init.py", line 1, in
from .inference import (convert_SyncBN, inference_detector, init_detector,
File "/root/MSMDFusion-main/mmdet3d/apis/inference.py", line 8, in
from mmdet3d.core import Box3DMode, show_result
File "/root/MSMDFusion-main/mmdet3d/core/init.py", line 1, in
from .anchor import * # noqa: F401, F403
File "/root/MSMDFusion-main/mmdet3d/core/anchor/init.py", line 1, in
from mmdet.core.anchor import build_anchor_generator
File "/root/MSMDFusion-main/mmdetection-2.x/mmdet/core/init.py", line 3, in
from .bbox import * # noqa: F401, F403
File "/root/MSMDFusion-main/mmdetection-2.x/mmdet/core/bbox/init.py", line 8, in
from .samplers import (BaseSampler, CombinedSampler,
File "/root/MSMDFusion-main/mmdetection-2.x/mmdet/core/bbox/samplers/init.py", line 12, in
from .score_hlr_sampler import ScoreHLRSampler
File "/root/MSMDFusion-main/mmdetection-2.x/mmdet/core/bbox/samplers/score_hlr_sampler.py", line 3, in
from mmcv.ops import nms_match
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/mmcv/ops/init.py", line 2, in
from .assign_score_withk import assign_score_withk
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/mmcv/ops/assign_score_withk.py", line 6, in
'_ext', ['assign_score_withk_forward', 'assign_score_withk_backward'])
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "/root/anaconda3/envs/MSMDFusion/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /root/anaconda3/envs/MSMDFusion/lib/python3.7/site-packages/mmcv/_ext.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops19empty_memory_format4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEENS6_INS2_12MemoryFormatEEE
'
Hi!
Due to the use of the offline virtual points, I wonder whether this code is applicable to dataset: nuscenes-mini?
Hope for your response!
Best
Hi!
Do you have this .json file. I don't have enough GPU to generete it myself, I would really appreciate it if you could provide if for me :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.