lizhaoliu-lec / cpcm Goto Github PK

28.0 3.0 3.0 6 MB

This is the official repo for Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation (ICCV 23).

License: MIT License

Python 95.84% Shell 4.16%

3d-segmentation augmentation consistency-regularization context-model data-efficient-learning point-cloud pytorch-implementation segmentation semi-supervised-learning sparse-annotations

cpcm's Introduction

Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation (ICCV 2023)

Created by Lizhao Liu, Xunlong Xiao, Zhuangwei Zhuang from the South China University of Technology.

This repository contains the official PyTorch implementation of our ICCV 2023 paper Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation.

Environment Setup

Our codebase is based on MinkowskiEngine, a high performance sparse convolution library built on PyTorch.

We recommend to use MinkowskiEngine 0.5.4, since it is much faster than 0.4.3

For MinkowskiEngine 0.5.4, see instruction in me054

For MinkowskiEngine 0.4.3, see instruction in me043

Data Preparation

We perform experiments on the following dataset

The preprocessed datasets are shared via google drive

Or see instruction in Dataset Preparation Hand-by-hand to prepare by yourself.

Quantitative Results

All results below are in mIoU(%)

Experiments on the indoor dataset: ScanNet V2 and S3DIS

Method	ScanNet V2		S3DIS
Method	0.01%	0.1%	0.01%	0.1%
MinkNet	37.6	60.3	47.7	62.9
Consis-based	44.2 (+6.6)	61.8 (+1.5)	52.9 (+5.2)	64.9 (+2.0)
CPCM (Ours)	52.2 (+14.6)	63.8 (+3.5)	59.3 (+11.6)	66.3 (+3.4)

Experiments on the outdoor dataset SemanticKITTY (FoV)

Method	SemanticKITTY
Method	1%	0.1%	0.01%
MinkNet	37.0	30.8	23.7
Consis-based	43.7 (+6.7)	38.8 (+8.0)	30.0 (+6.3)
CPCM (Ours)	47.8 (+10.8)	44.0 (+13.2)	34.7 (+11.0)

Qualitative Results for ScanNet and S3DIS

The first two rows are results for ScanNet, the last two rows are results for S3DIS

Experiments on S3DIS

To reproduce the results of S3DIS, see experiment scripts here for details.

Experiments on ScanNet V2

To reproduce the results of ScanNet V2, see experiment scripts here for details. The script that generates ScanNet testset results are also available here.

Experiments on SemanticKITTY (FoV)

To reproduce the results of ScanNet V2, see experiment scripts here for details.

Acknowledgement

This codebase is partially built on the PointContrast project.

Citation

If you find this code helpful for your research, please consider citing

@inproceedings{liu2023contextual,
  title={CPCM: Contextual point cloud modeling for weakly-supervised point cloud semantic segmentation},
  author={Liu, Lizhao and Zhuang, Zhuangwei and Huang, Shangxin and Xiao, Xunlong and Xiang Tianhang and Chen, Cen and Wang, Jingdong and Tan, Mingkui},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2023}
}

cpcm's People

Contributors

Stargazers

Watchers

Forkers

liuxinren456852 cv-seg pointcloudyc

cpcm's Issues

How to visualise the results after testing?

Thank you for such a good job. I'm a little confused. How do you visualise your test results?Looking forward to your reply.

Cannot reproduce the results

Thanks for the great research, I'm having some issues running this. MinkowskiEngine 0.5.4 is used. During the process of preparing S3DIS data, an error occurred and the return value was incorrect. However, it can run normally after subsequent modifications. However, when running CPCM's 0.1S3DIS data alone, val_miou is 56%, and both the script and default.yaml are the latest in the project. Looking forward to your reply。

how to prepare weak labels of S3DIS dataset?

Hi? Could you please share scripts for preparing weak labels of the S3DIS dataset?
Thanks.

question about multi-GPUs training

thank you for your excellent work.
because my sigle gpu's memory isn't enough, I want to know if your code can be modified use multi-gpus to train.

can you provide 0.02% S3DIS experiment scripts？

What type of data is percentage0.001evenc?

I want to know the data type of percentage0.001evenc and how to open it，thank you

How to test?

Thank you for such a good job. I have a small question. Trained, how to test. Which parameters should be changed?

Cannot reproduce the results.

Thank you for your codes. I have found that the setting of two_stream_mask_self_loss in default.yaml is "False", which means that the calculation of mask loss for feats_aux and feats_masked_aux (Z2, Zm loss in your paper) is not involved. I also found that two_stream_mask_mode was set to "random", while it should be "grid". I have corrected them and test under 0.1% setting on S3DIS, while the mIoU is only 63.5, not 66.3.

Question about the requirement of GPU

Hi! I wonder how many GPUs and what kind of GPUs are required in general weakly-supervised semantic segmentation task including your work? Plus as introduced in your paper, 1 or 2 TITAN 3090 GPU is enough, so how long does it take for a single group of experiment? Thanks!

Dose your code only can be train on MinkowskiEngine 0.4.3 Env now

thank you for your excellent work. my hardware are 2080ti *5,may be I can't use MinkowskiEngine 0.5.4.
Dose your code only can be train on MinkowskiEngine 0.4.3 Env now,beause I see this:

Question about the training script for the 0.02% (S3DIS)

Hi.
Thank you for releasing the code.
However, I cannot find the training script for the 0.02% setting on the S3DIS dataset.
I modified the script for 0.01% as 0.02% and tried it.
Unfortunately, this setting fails to reproduce the reported performance.
Could you provide the official training script for it?
Any helpful comment will be greatly appreciated.

merge_sort: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered terminate called after throwing an instance of 'c10::Error'

Thanks for your code! I have encountered an error when training:

Traceback (most recent call last):
File "ddp_train.py", line 116, in
main()
File "ddp_train.py", line 107, in main
trainer.train()
File "/home/Point_Cloud/CPCM/trainer/base.py", line 190, in train
self.train_one_epoch()
File "/home/Point_Cloud/CPCM/trainer/fully_supervised_trainer.py", line 332, in train_one_epoch
step_ret = self.step(batch)
File "/home/Point_Cloud/CPCM/trainer/fully_supervised_trainer.py", line 1232, in step
return self._step_two_and_mask_stream(batch=batch)
File "/home/Point_Cloud/CPCM/trainer/fully_supervised_trainer.py", line 1199, in _step_two_and_mask_stream
loss.backward()
File "/home/.conda/envs/seg/lib/python3.8/site-packages/torch/_tensor.py", line 307, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/.conda/envs/seg/lib/python3.8/site-packages/torch/autograd/init.py", line 154, in backward
Variable._execution_engine.run_backward(
RuntimeError: merge_sort: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered
terminate called after throwing an instance of 'c10::Error'

How can I fix this? I tried this taesungp/contrastive-unpaired-translation#83 but did not success.