allenwu97 / edam Goto Github PK

The code and pretrain models of EDAM-WSSS(Embedded Discriminative Attention Mechanism for Weakly Supervised Semantic Segmentation)

Python 100.00%

edam wsss

edam's Introduction

Embedded Discriminative Attention Mechanism for Weakly Supervised Semantic Segmentation(EDAM-WSSS) CVPR2021

The code and pretrain models of EDAM(Embedded Discriminative Attention Mechanism for Weakly Supervised Semantic Segmentation) Our papaer link

Env

We train our model with Python 3.5, PyTorch 1.1.0 and 4 Tesla V100 GPUs with 16 GB memory. Other Python modules can be installed by running

conda install --yes --file requirements.txt

You can also download our env directly [BaiduYun] tlu7

After unzipping, running

source py3.5/bin/activate

Dataset

The PASCAL VOC 2012 development kit: You also need to specify the path ('voc12_root') of your downloaded dev kit.
Saliency Maps [GoogleDrive][BaiduYun] (fjst): Class-agnostic saliency maps generated by PoolNet as background cues.

Pretrain Models

We provide three pre-trained models, including initialization parameters, EDAM parameters and Deeplabv2 parameters [Google Drive][BaiduYun] u7q7

Train

Training EDAM from scratch.(Noting: We have pre-processed the dataset, and the amount of data in each epoch is equivalent to the original 25 epochs)

python3 train_EDAM_cls.py --lr 0.001 --batch_size 4 --max_epoches 1 --crop_size 368 --network network.resnet38_EDAM_cls --voc12_root [Root of VOC12] --weights [Root of initialization parameters] --wt_dec 5e-4 --session_name resnet38_EDAM_cls

To monitor loss and lr, run the following command in a separate terminal.

tensorboard --logdir runs

Test

Generate pseudo labels.

python3 infer_cls.py --infer_list voc12/train_aug.txt --voc12_root /workdir/VOCdevkit/VOC2012 --network network.resnet38_EDAM_cls --weights [Path of EDAM Parameters]  --out_crf_pred [Output Path] --theta 0.2  --alpha 0.000001  --beta 0.99999 --sal_path [Path of Saliency Map]

Vis pseudo labels

python3 colorful.py --img_path [Path of Pseudo Labels] —out_path [Path of Colorful Pseudo Labels]

Segmentation Network

We use our pseudo labels fully-supervised train a Deeplab-v2 Network
We also provide the final pseudo labels for segmentation network training. [Google Drive][BaiduYun] 9aij

Results

Model	Train set	Val set	Crf?	Saliency?	Mean IoU
EDAM	trainaug	train	-	-	52.83
			✓	-	58.61
			✓	✓	68.11
DeepLab-v2	trainaug	val	-	-	69.66
			✓	-	70.96

Citation

If this code is useful to you, consider using the following citation:

@InProceedings{Wu_2021_CVPR,
    author    = {Wu, Tong and Huang, Junshi and Gao, Guangyu and Wei, Xiaoming and Wei, Xiaolin and Luo, Xuan and Liu, Chi Harold},
    title     = {Embedded Discriminative Attention Mechanism for Weakly Supervised Semantic Segmentation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {16765-16774}
}

edam's People

Contributors

Stargazers

Watchers

Forkers

sunwanchun marilynmansom niexiuping mymuli

edam's Issues

About forward_cam

def forward_cam(self, x):
    x = super().forward(x)
    mask = self.mask_layer(x)
    mask = F.normalize(mask)
    mask = torch.abs(mask)
    return mask

=====================
Dear authors, I consider it might be unfair to use 'torch.abs(mask)'. Because the negative values corresponding to under-activated regions would be directly transformed into postive values. In fact, 'torch.relu(mask)' was used in previous works.

Questions about the test result

Your EDAM is the experimental result done with the provided weights and the provided train set? (Mean IoU: 68.11) According to your test steps, I only get Mean IoU: 13.29, which part do I Wrong? Hope you can clear my doubts, thanks.

python3 infer_cls.py --infer_list voc12/train.txt --voc12_root ./VOC2012 --network network.resnet38_EDAM_cls --weights ./pretrain_model/download.params --out_cam_pred cam_pred --out_crf_pred out_crf_pred --theta 0.2 --alpha 0.0025 --beta 0.0095 --sal_path ./saliency_map_train_aug

python3 eval.py --predict_dir out_crf_pred

how can i get the paper?

About weight downloads

Hi, thanks for your work.
Regarding weights, do you have any plans to share them outside of Baidu?
It is difficult for foreigners to do downloads via Baidu.
(This is the first time I've asked a question in this kind of forum, so I apologize if I'm being rude.)

Questions about pretrained models of DeepLab.

Hi, thanks for your work.
I wonder which pre-trained models you use for DeepLab.

The file of requirement.txt does not exist

Why the file of requirement.txt does not exist in the code file?

Question about Algorithm 1

Hi, thanks for your work. I have a question about Algorithm 1.

According to the paper, alpha=0.25, beta=0.95, but in this repo, alpha=0.00001, beta=0.99999

Thanks.

Questions about DeepLab

Hi, thanks for your work. I have some questions about DeepLab.

According to the paper, DeepLab-LargeFOV is used as the segmentation network, however, Deeplab-aspp is used in this project.
Could you please provide more details on training segmentation network.

Thanks.

from prefetch_generator import BackgroundGenerator?Can find your implementation?

The dataset of web and caltech-256

Hi, nice work!

Could you please offer the web and caltech-256 that your used, including images and image level labels?

Thank you very much!

Questions about the resnet38_EDAM_cls.py

Hi @allenwu97 ! Thansk for sharing your nice work!

I want to reproduce your work but meet the error of "AssertionError" in line 55 of resnet38_EDAM_cls.py : "assert n == self.mini_batch_size', when I run your code with your default command line:
"python3 train_EDAM_cls.py --lr 0.001 --batch_size 4 --max_epoches 1 --crop_size 368 --network network.resnet38_EDAM_cls --voc12_root [Root of VOC12] --weights [Root of initialization parameters] --wt_dec 5e-4 --session_name resnet38_EDAM_cls
"
Then I try to solve this problem and remove the above line, but another problem appears:"RuntimeError: The size of tensor a (128) must match the size of tensor b (2) at non-singleton dimension 0" in line 66 of resnet38_EDAM_cls.py.

I notice your dataloader is different from other WSSS released code. I don't know how to do with it, can you help me ? Thanks a lot!