kinpzz / rcrnet-pytorch Goto Github PK

Semi-Supervised Video Salient Object Detection Using Pseudo-Labels, IEEE International Conference on Computer Vision (ICCV), 2019

Home Page: https://kinpzz.com/publication/iccv19_semi_vsod/

License: MIT License

Dockerfile 0.17% Python 78.72% Shell 0.97% C++ 3.33% Cuda 16.81%

video-object-segmentation video-saliency iccv2019 pytorch

rcrnet-pytorch's Introduction

RCRNet-Pytorch

This repository contains the PyTorch implementation for

Semi-Supervised Video Salient Object Detection Using Pseudo-Labels
Pengxiang Yan, Guanbin Li, Yuan Xie, Zhen Li, Chuan Wang, Tianshui Chen, Liang Lin
ICCV 2019 | [Project Page] | [Arxiv] | [CVF-Open-Access]

Usage

Requirements

This code is tested on Ubuntu 16.04, Python=3.6 (via Anaconda3), PyTorch=0.4.1, CUDA=9.0.

# Install PyTorch=0.4.1
$ conda install pytorch==0.4.1 torchvision==0.2.1 cuda90 -c pytorch

# Install other packages
$ pip install pyyaml==3.13 addict==2.2.0 tqdm==4.28.1 scipy==1.1.0

Datasets

Our proposed RCRNet is evaluated on three public benchmark VSOD datsets including VOS, DAVIS (version: 2016, 480p), and FBMS. Please orginaize the datasets according to config/datasets.yaml and put them in data/datasets. Or you can set argument --data to the path of the dataset folder.

Evaluation

Comparison with State-of-the-Art

If you want to compare with our method:

Option 1: you can download the saliency maps predicted by our model from Google Drive / Baidu Pan (passwd: u079).

Option 2: Or you can use our trained model for inference. The weights of trained model are available at Google Drive / Baidu Pan (passwd: 6pi3). Then run the following command for inference.

# VOS
$ CUDA_VISIBLE_DEVICES=0 python inference.py --data data/datasets --dataset VOS --split test

# DAVIS
$ CUDA_VISIBLE_DEVICES=0 python inference.py --data data/datasets --dataset DAVIS --split val

# FBMS
$ CUDA_VISIBLE_DEVICES=0 python inference.py --data data/datasets --dataset FBMS --split test

Then, you can evaluate the saliency maps using your own evaluation code.

Training

If you want to train our proposed model from scratch (including using pseudo-labels), please refer to our paper and the training instruction carefully.

Citation

If you find this work helpful, please consider citing

@inproceedings{yan2019semi,
  title={Semi-Supervised Video Salient Object Detection Using Pseudo-Labels},
  author={Yan, Pengxiang and Li, Guanbin and Xie, Yuan and Li, Zhen and Wang, Chuan and Chen, Tianshui and Lin, Liang},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  pages={7284--7293},
  year={2019}
}

Acknowledge

Thanks to the third-party libraries:

deeplab-pytorch by kazuto1011
flownet2-pytorch by NVIDIA
pytorch-segmentation-toolbox by speedinghzl
Non-local_pytorch by AlexHex7

rcrnet-pytorch's People

Contributors

Stargazers

Watchers

Forkers

wolfworld6 yangshushuaige jongchan lhyciomp snlee81 kkthej wanghongyu156 saliencydetection jodyngo luweishuang menghah wpfhtl xueliancheng ybwang0820 f200ten tommywhy stddef

rcrnet-pytorch's Issues

请教一下VOS数据集配置

前辈您好，我在复现您RCRNet的结果时，基本上没怎么改您的代码，用您训练好的伪标签生成器每5帧生成1帧伪标签，目前DAVIS和FBMS数据集上的性能都差不多，但是VOS 的 test 数据集的性能差了5-6个点。后来干脆不用伪标签，直接将伪标签生成器的frame_between_label_num设置为0，这样的话，相当于直接生成的是20%的真值。我用这个训练，VOS test数据集的指标还是差了5-6个点。但是用您提供的best_model直接跑inference，VOS的指标又是一样的。目前猜测是VOS文件配置问题？
DAVIS数据集配置：JPEGImages是帧间隔为1，伪标签文件夹里的标签（真值）帧间隔为5
FBMS数据集配置：JPEGImages是帧间隔不定，对应原始100%真值的图（一般间隔为20帧），伪标签文件夹里的标签（真值）帧间隔再乘以 5
VOS数据集配置：JPEGImages是帧间隔为1，伪标签文件夹里的标签（真值）帧间隔为15 x 5 =75

我不太确定到底是哪里错了，能帮我对一下VOS数据集配置有问题吗

the Same paper as "Real-time Segmenting Human Portrait at Anywhere" ?

Real-time Segmenting Human Portrait at Anywhere
Ruifeng Yuan, Yuhao Cheng♯
, Yiqiang Yan, Haiyan Liu
Lenovo Research
Buidling1, No.10 Courtyard Xibeiwang East Road, Beijing, China

I found it use RCRNet too?

but "Real-time Segmenting Human Portrait at Anywhere" has no github traning code

RuntimeError: CUDA call failed (correlation_forward_cuda at correlation_cuda.cc:82)

RuntimeError: CUDA call failed (correlation_forward_cuda at correlation_cuda.cc:82)
According to the readme.md, we changed the same configuration. When we run generate_pseudo_labels.py, the program appears above error, could you please help us to solve this problem?