Efficient Spatio-Temporal Recurrent Neural Network for Video Deblurring (ECCV2020 Spotlight)

by Zhihang Zhong, Ye Gao, Yinqiang Zheng, Bo Zheng

• Python 3.6
• PyTorch 1.6 with GPU
• opencv-python
• scikit-image
• lmdb
• thop
• tqdm
• tensorboard

We have collected a new BSD dataset with more scenes and better setups (center-aligned), using the proposed beam-splitter acquisition system:

The configurations of the new BSD dataset are as below:

Results on different setups of BSD:

Pretrained models for different setups:

Please download checkpoints and unzip it under the main directory.

Example command to run a pre-trained model:

python main.py --test_only --test_checkpoint ./checkpoints/ESTRNN_C80B15_BSD_3ms24ms.tar --ds_config 3ms24ms --video

Please download and unzip the dataset file for each benchmark.

• BSD
• GOPRO-DS
• REDS

Then, specify the <path> (e.g. "./dataset/ ") where you put the dataset file and the corresponding dataset configurations in the command, or change the default values in "./para/paramter.py".

Training command is as below:

python main.py --data_root <path> --dataset BSD --ds_config 2ms16ms --data_format RGB

You can also tune the hyper parameters such as batch size, learning rate, epoch number, etc. (P.S.: the actual batch size for ddp mode is num_gpus*batch_size)

python main.py --lr 1e-4 --batch_size 4 --num_gpus 2 --trainer_mode ddp

If you want to train on your own dataset, please refer to "/data/how_to_make_dataset_file.ipynb".

If you use any part of our code, or ESTRNN and BSD are useful for your research, please consider citing:

@inproceedings{zhong2020efficient,
  title={Efficient spatio-temporal recurrent neural network for video deblurring},
  author={Zhong, Zhihang and Gao, Ye and Zheng, Yinqiang and Zheng, Bo},
  booktitle={European Conference on Computer Vision},
  pages={191--207},
  year={2020},
  organization={Springer}
}