This repository provides the official code for the paper Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation.
🥳 We are happy to announce that RIPU was accepted at CVPR 2022 Oral Presentation.
We propose a simple region-based active learning approach for semantic segmentation under a domain shift, aiming to automatically query a small partition of image regions to be labeled while maximizing segmentation performance. Our algorithm, RIPU, introduces a new acquisition strategy characterizing the spatial adjacency of image regions along with the prediction confidence. The proposed region-based selection strategy makes more efficient use of a limited budget than image-based or point-based counterparts.
We show some qualitative examples from the Cityscapes validation set,
and also visualize the queried regions to annotate.
For more information on RIPU, please check our Paper.
- Python 3.7
- Pytorch 1.7.1
- torchvision 0.8.2
Step-by-step installation
conda create --name ADASeg -y python=3.7
conda activate ADASeg
# this installs the right pip and dependencies for the fresh python
conda install -y ipython pip
# this installs required packages
pip install -r requirements.txt
- Download The Cityscapes Dataset, The GTAV Dataset, and The SYNTHIA Dataset
Symlink the required dataset
ln -s /path_to_cityscapes_dataset datasets/cityscapes
ln -s /path_to_gtav_dataset datasets/gtav
ln -s /path_to_synthia_dataset datasets/synthiaGenerate the label static files for GTAV/SYNTHIA Datasets by running
python datasets/generate_gtav_label_info.py -d datasets/gtav -o datasets/gtav/
python datasets/generate_synthia_label_info.py -d datasets/synthia -o datasets/synthia/The data folder should be structured as follows:
├── datasets/
│ ├── cityscapes/
| | ├── gtFine/
| | ├── leftImg8bit/
│ ├── gtav/
| | ├── images/
| | ├── labels/
| | ├── gtav_label_info.p
│ └── synthia
| | ├── RAND_CITYSCAPES/
| | ├── synthia_label_info.p
│ └──
We have put our model checkpoints here [Google Drive] [百度网盘] (提取码RIPU).
| name | backbone | budget | mIoU | ckpt | where in Our Paper | |
| 1 | RIPU-PA | V2 | 40 px | 65.5 | Google Drive / BaiDu | Table 1 |
| 2 | RIPU-RA | V2 | 2.2% | 69.6 | Google Drive / BaiDu | Table 1 |
| 3 | RIPU-RA | V3+ | 5.0% | 71.2 | Google Drive / BaiDu | Table 1 |
| name | backbone | budget | mIoU | ckpt | where in Our Paper | |
| 1 | RIPU-PA | V2 | 40 px | 66.1 | Google Drive / BaiDu | Table 2 |
| 2 | RIPU-RA | V2 | 2.2% | 70.1 | Google Drive / BaiDu | Table 2 |
| 3 | RIPU-RA | V3+ | 5.0% | 71.4 | Google Drive / BaiDu | Table 2 |
| task | budget | mIoU | source pre-trained ckpt | adapted ckpt | Where in Our Paper | |
| 1 | GTAV to Cityscapes | 2.2% | 67.1 | Google Drive / BaiDu | Google Drive / BaiDu | Table 12 |
| 2 | SYNTHIA to Cityscapes | 2.2% | 68.7 | Google Drive / BaiDu | Google Drive / BaiDu | Table 13 |
We provide the training scripts in scripts/ using a single GPU.
# training for GTAV to Cityscapes
sh gtav_to_cityscapes.sh
# training for SYNTHIA to Cityscapes
sh synthia_to_cityscapes.shTo evaluate RIPU e.g. GTAV to Cityscapes (v3+, 5.0%), use the following command:
python test.py -cfg configs/gtav/deeplabv3plus_r101_RA.yaml resume checkpint/v3plus_gtav_ra_5.0_precent/model_last.pth OUTPUT_DIR checkpint/v3plus_gtav_ra_5.0_precentThis project is based on the following open-source projects: FADA and SDCA. We thank their authors for making the source code publically available.
If you find this project useful in your research, please consider citing:
@InProceedings{xie2022ripu,
author = {Xie, Binhui and Yuan, Longhui and Li, Shuang and Liu, Chi Harold and Cheng, Xinjing},
title = {Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2022},
pages = {8068-8078}
}If you have any problem about our code, feel free to contact
or describe your problem in Issues.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent