The frnet from zwilsonss

FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation

Xiang Xu¹,   Lingdong Kong²,   Hui Shuai³,   Qingshan Liu³
¹Nanjing University of Aeronautics and Astronautics    ²National University of Singapore
³Nanjing University of Posts and Telecommunications

About

FRNet is a simple yet efficient network for LiDAR segmentation. It consists of three key components: 1) Frustum Feature Encoder; 2) Frustum-Point Fusion Module; and 3) Head Fusion Module. Moreover, we introduce two novel argumentations, FrustumMix and RangeInterpolation, to enrich the point cloud scenes.

FRNet achieves an appealing balance between accuracy and efficiency, enabling real-time LiDAR segmentation.


Speed vs. Accuracy	Speed vs. Robustness

Visit our project page to explore more examples. 🚗

Updates

[2024.04] - We have further improved our manuscript and code.
[2023.12] - We provide trained weights on SemanticKITTI and nuScenes. The checkpoints can be downloaded from here.
[2023.12] - Our paper is available on arXiv, and the code is publicly available.

🎥 Video Demo

Demo 1	Demo 2	Demo 3

[Link]	[Link]	[Link]

⚙️ Installation

Please refer to INSTALL.md for the installation details.

♨️ Data Preparation

Please refer to DATA_PREPARE.md for the details to prepare the ¹SemanticKITTI and ²nuScenes datasets.

🚀 Getting Started

Please refer to GET_STARTED.md to learn more usage of this codebase.

📊 Main Results

Framework Overview

Comparisons to State of the Arts

Method	Param	SemanticKITTI			nuScenes		ScribbleKITTI		SemanticPOSS
Method	Param	FPS	Val	Test	Val	Test	mIoU	mAcc	mIoU	mAcc
CENet	6.8 M	33.4	62.6	64.7	-	-	55.7	66.8	50.3	-
RangeViT	23.7 M	10.0	60.7	64.0	75.2	-	53.6	66.5	-	-
Fast-FRNet	7.5 M	33.8	67.1	72.5	78.8	82.1	62.4	71.2	52.5	67.1
FRNet	10.0 M	29.1	68.7	73.3	79.0	82.5	63.1	72.3	53.5	68.1

Label-Efficient LiDAR Segmentation

Method	SemanticKITTI				nuScenes				ScribbleKITTI
Method	1%	10%	20%	50%	1%	10%	20%	50%	1%	10%	20%	50%
Sup.-only	44.9	60.4	61.8	63.1	51.9	68.1	70.9	74.6	42.4	53.5	55.1	57.0
LaserMix	52.9	62.9	63.2	65.0	58.7	71.5	72.3	75.0	45.8	56.8	57.7	59.0
FrustumMix	55.8	64.8	65.2	65.4	61.2	72.2	74.6	75.4	46.6	57.0	59.5	61.2

Robustness

Method	SemKITTI-C		nuScenes-C
Method	mCE	mRR	mCE	mRR
CENet	103.4	81.3	112.8	76.0
FRNet	96.8	80.0	98.6	77.5

📝 Note:

mCE (the lower the better): The average corruption error (in percentage) of a candidate model compared to the baseline model, which is calculated among all corruption types across three severity levels.
mRR (the higher the better): The average resilience rate (in percentage) of a candidate model compared to its "clean" performance, which is calculated among all corruption types across three severity levels.

📍 Pre-Trained Checkpoints

We provide the trained models for SemanticKITTI and nuScenes. The checkpoints can be downloaded from here.

License

This work is under the Apache 2.0 license.

Citation

If you find this work helpful, please kindly consider citing our paper:

@article{xu2023frnet,
    title = {FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation},
    author = {Xu, Xiang and Kong, Lingdong and Shuai, Hui and Liu, Qingshan},
    journal = {arXiv preprint arXiv:2312.04484},
    year = {2023}
}

Acknowledgements

This work is developed based on the MMDetection3D codebase.

MMDetection3D is an open-source object detection toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

We acknowledge the use of the following public resources during the course of this work: ¹SemanticKITTI, ²SemanticKITTI-API, ³nuScenes, ⁴nuScenes-devkit, ⁵ScribbleKITTI, ⁶SemanticPOSS, ⁷SemanticPOSS-API, ⁸Robo3D, ⁹PCSeg, ¹⁰SalsaNext, ¹¹FIDNet, ¹²CENet, ¹³RangeViT, ¹⁴SphereFormer, ¹⁵2DPASS, ¹⁶Cylinder3D, ¹⁷SPVNAS, ¹⁸KPConv, and ¹⁹LaserMix. 💟

zwilsonss / frnet Goto Github PK

frnet's Introduction