EfficientPS is a state-of-the-art top-down approach for panoptic segmentation, where the goal is to assign semantic labels (e.g., car, road, tree and so on) to every pixel in the input image as well as instance labels (e.g. an id of 1, 2, 3, etc) to pixels belonging to thing classes.

This repository contains the PyTorch implementation of our IJCV'2021 paper EfficientPS: Efficient Panoptic Segmentation. The repository builds on mmdetection and gen-efficientnet-pytorch codebases.

If you find the code useful for your research, please consider citing our paper:

@article{mohan2020efficientps,
  title={Efficientps: Efficient panoptic segmentation},
  author={Mohan, Rohit and Valada, Abhinav},
  journal={International Journal of Computer Vision (IJCV)},
  year={2021}
}

http://rl.uni-freiburg.de/research/panoptic

• Linux
• Python 3.7
• PyTorch 1.4
• CUDA 10.2
• GCC 7 or 8

IMPORTANT NOTE: These requirements are not necessarily mandatory. However, we have only tested the code under the above settings and cannot provide support for other setups.

a. Create a conda virtual environment from the provided environment.yml and activate it.

git clone https://github.com/DeepSceneSeg/EfficientPS.git
cd EfficientPS
conda env create -n efficientPS_env --file=environment.yml
conda activate efficientPS_env

b. Install all other dependencies using pip:

pip install -r requirements.txt

c. Install EfficientNet implementation

cd efficientNet
python setup.py develop

d. Install EfficientPS implementation

cd ..
python setup.py develop

It is recommended to symlink the dataset root to $EfficientPS/data. If your folder structure is different, you may need to change the corresponding paths in config files.

EfficientPS
├── mmdet
├── tools
├── configs
├── data
│   ├── cityscapes
│   │   ├── annotations
│   │   ├── train
│   │   ├── val
│   │   ├── stuffthingmaps
│   │   ├── cityscapes_panoptic_val.json
│   │   ├── cityscapes_panoptic_val

The cityscapes annotations have to be converted into the aforementioned format using tools/convert_datasets/cityscapes.py:

python tools/convert_cityscapes.py ROOT_DIRECTORY_OF_CITYSCAPES ./data/cityscapes/
cd ..
git clone https://github.com/mcordts/cityscapesScripts.git
cd cityscapesScripts/cityscapesscripts/preparation
python create createPanopticImgs.py --dataset-folder ROOT_DIRECTORY_OF_CITYSCAPES --output-folder ../../../EfficientPS/data/cityscapes --set-names val

Edit the config file appropriately in configs folder. Train with a single GPU:

python tools/train.py efficientPS_singlegpu_sample.py --work_dir work_dirs/checkpoints --validate 

Train with multiple GPUS:

./tools/dist_train.sh efficientPS_multigpu_sample.py ${GPU_NUM} --work_dir work_dirs/checkpoints --validate 

• --resume_from ${CHECKPOINT_FILE}: Resume from a previous checkpoint file.

Test with a single GPU:

python tools/test.py efficientPS_singlegpu_sample.py ${CHECKPOINT_FILE} --eval panoptic

Test with multiple GPUS:

./tools/dist_test.sh efficientPS_multigpu_sample.py ${CHECKPOINT_FILE} ${GPU_NUM} --eval panoptic

Coming Soon !!!

• We only provide the single scale evaluation script. Multi-Scale+Flip evaluation further imporves the performance of the model.
• This is a re-implementation of EfficientPS in PyTorch. The performance of the trained models might slightly differ from the metrics reported in the paper. Please refer to the metrics reported in EfficientPS: Efficient Panoptic Segmentation when making comparisons.

We have used utility functions from other open-source projects. We espeicially thank the authors of:

• mmdetection
• gen-efficientnet-pytorch
• seamseg

• Abhinav Valada
• Rohit Mohan

For academic usage, the code is released under the GPLv3 license. For any commercial purpose, please contact the authors.