This repo extends the PointNet++ model for semantic segmentation of buildings. The model is trained on a portion of the 2022 Vancouver LiDAR Dataset whereby the points are filtered by their proximity to a building (points that are within 5m of a building) as a pre-processing step. This implementation is developed as a part of the LiDAR-DenseSeg project.
Pre-trained weights are available in the root directory of this repo and also under the checkpoints/
directory.
On the UBC dataset, the model achieves an accuracy of 94.1% and a mean IoU of 0.89%.
- Further hyperparameter tuning and evaluation on a larger dataset is planned (extrapolation to a larger area of the Vancouver LiDAR dataset).
- Randomization of building coordinates during training process to improve generalization to unseen buildings and other datasets.
The following is the original README for the PointNet++ model implementation in PyTorch.
This repo is implementation for PointNet and PointNet++ in pytorch.
2021/03/27:
(1) Release pre-trained models for semantic segmentation, where PointNet++ can achieve 53.5% mIoU.
(2) Release pre-trained models for classification and part segmentation in log/
.
2021/03/20: Update codes for classification, including:
(1) Add codes for training ModelNet10 dataset. Using setting of --num_category 10
.
(2) Add codes for running on CPU only. Using setting of --use_cpu
.
(3) Add codes for offline data preprocessing to accelerate training. Using setting of --process_data
.
(4) Add codes for training with uniform sampling. Using setting of --use_uniform_sample
.
2019/11/26:
(1) Fixed some errors in previous codes and added data augmentation tricks. Now classification by only 1024 points can achieve 92.8%!
(2) Added testing codes, including classification and segmentation, and semantic segmentation with visualization.
(3) Organized all models into ./models
files for easy using.
The latest codes are tested on Ubuntu 16.04, CUDA10.1, PyTorch 1.6 and Python 3.7:
conda install pytorch==1.6.0 cudatoolkit=10.1 -c pytorch
Download alignment ModelNet here and save in data/modelnet40_normal_resampled/
.
You can run different modes with following codes.
- If you want to use offline processing of data, you can use
--process_data
in the first run. You can download pre-processd data here and save it indata/modelnet40_normal_resampled/
. - If you want to train on ModelNet10, you can use
--num_category 10
.
# ModelNet40
## Select different models in ./models
## e.g., pointnet2_ssg without normal features
python train_classification.py --model pointnet2_cls_ssg --log_dir pointnet2_cls_ssg
python test_classification.py --log_dir pointnet2_cls_ssg
## e.g., pointnet2_ssg with normal features
python train_classification.py --model pointnet2_cls_ssg --use_normals --log_dir pointnet2_cls_ssg_normal
python test_classification.py --use_normals --log_dir pointnet2_cls_ssg_normal
## e.g., pointnet2_ssg with uniform sampling
python train_classification.py --model pointnet2_cls_ssg --use_uniform_sample --log_dir pointnet2_cls_ssg_fps
python test_classification.py --use_uniform_sample --log_dir pointnet2_cls_ssg_fps
# ModelNet10
## Similar setting like ModelNet40, just using --num_category 10
## e.g., pointnet2_ssg without normal features
python train_classification.py --model pointnet2_cls_ssg --log_dir pointnet2_cls_ssg --num_category 10
python test_classification.py --log_dir pointnet2_cls_ssg --num_category 10
Model | Accuracy |
---|---|
PointNet (Official) | 89.2 |
PointNet2 (Official) | 91.9 |
PointNet (Pytorch without normal) | 90.6 |
PointNet (Pytorch with normal) | 91.4 |
PointNet2_SSG (Pytorch without normal) | 92.2 |
PointNet2_SSG (Pytorch with normal) | 92.4 |
PointNet2_MSG (Pytorch with normal) | 92.8 |
Download alignment ShapeNet here and save in data/shapenetcore_partanno_segmentation_benchmark_v0_normal/
.
## Check model in ./models
## e.g., pointnet2_msg
python train_partseg.py --model pointnet2_part_seg_msg --normal --log_dir pointnet2_part_seg_msg
python test_partseg.py --normal --log_dir pointnet2_part_seg_msg
Model | Inctance avg IoU | Class avg IoU |
---|---|---|
PointNet (Official) | 83.7 | 80.4 |
PointNet2 (Official) | 85.1 | 81.9 |
PointNet (Pytorch) | 84.3 | 81.1 |
PointNet2_SSG (Pytorch) | 84.9 | 81.8 |
PointNet2_MSG (Pytorch) | 85.4 | 82.5 |
Download 3D indoor parsing dataset (S3DIS) here and save in data/s3dis/Stanford3dDataset_v1.2_Aligned_Version/
.
cd data_utils
python collect_indoor3d_data.py
Processed data will save in data/stanford_indoor3d/
.
## Check model in ./models
## e.g., pointnet2_ssg
python train_semseg.py --model pointnet2_sem_seg --test_area 5 --log_dir pointnet2_sem_seg
python test_semseg.py --log_dir pointnet2_sem_seg --test_area 5 --visual
Visualization results will save in log/sem_seg/pointnet2_sem_seg/visual/
and you can visualize these .obj file by MeshLab.
halimacc/pointnet3
fxia22/pointnet.pytorch
charlesq34/PointNet
charlesq34/PointNet++
If you find this repo useful in your research, please consider citing it and our other works:
@article{Pytorch_Pointnet_Pointnet2,
Author = {Xu Yan},
Title = {Pointnet/Pointnet++ Pytorch},
Journal = {https://github.com/yanx27/Pointnet_Pointnet2_pytorch},
Year = {2019}
}
@InProceedings{yan2020pointasnl,
title={PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling},
author={Yan, Xu and Zheng, Chaoda and Li, Zhen and Wang, Sheng and Cui, Shuguang},
journal={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2020}
}
@InProceedings{yan2021sparse,
title={Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion},
author={Yan, Xu and Gao, Jiantao and Li, Jie and Zhang, Ruimao, and Li, Zhen and Huang, Rui and Cui, Shuguang},
journal={AAAI Conference on Artificial Intelligence ({AAAI})},
year={2021}
}
@InProceedings{yan20222dpass,
title={2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds},
author={Xu Yan and Jiantao Gao and Chaoda Zheng and Chao Zheng and Ruimao Zhang and Shuguang Cui and Zhen Li},
year={2022},
journal={ECCV}
}
- PointConv: Deep Convolutional Networks on 3D Point Clouds, CVPR'19
- On Isometry Robustness of Deep 3D Point Cloud Models under Adversarial Attacks, CVPR'20
- Label-Efficient Learning on Point Clouds using Approximate Convex Decompositions, ECCV'20
- PCT: Point Cloud Transformer
- PSNet: Fast Data Structuring for Hierarchical Deep Learning on Point Cloud
- Stratified Transformer for 3D Point Cloud Segmentation, CVPR'22