Code Monkey home page Code Monkey logo

csg's Introduction

Python 3.7

CSG: Contrastive Syn-to-Real Generalization

Paper

Contrastive Syn-to-Real Generalization.
Wuyang Chen, Zhiding Yu, Shalini De Mello, Sifei Liu, Jose M. Alvarez, Zhangyang Wang, Anima Anandkumar.
In ICLR 2021.

  • Visda-17 to COCO
  • train resnet101 with CSG
  • evaluation
  • GTA5 to Cityscapes
  • train deeplabv2 (resnet50/resnet101) with CSG
  • evaluation

Usage

Visda-17

Evaluation

python train.py \
--epochs 30 \
--batch-size 32 \
--lr 1e-4 \
--rand_seed 0 \
--csg 0.1 \
--apool \
--augment \
--csg-stages 3.4 \ 
--factor 0.1 \
--resume pretrained/csg_res101_vista17_best.pth.tar \
--evaluate
  • Run CUDA_VISIBLE_DEVICES=0 bash train.sh
    • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Train with CSG

  • Put the code below in train.sh
python train.py \
--epochs 30 \
--batch-size 32 \
--lr 1e-4 \
--rand_seed 0 \
--csg 0.1 \
--apool \
--augment \
--csg-stages 3.4 \ 
--factor 0.1 \
  • Run CUDA_VISIBLE_DEVICES=0 bash train.sh
    • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

GTA5 โ†’ Cityscapes

Evaluation

python train_seg.py \
--epochs 50 \
--switch-model deeplab50 \
--batch-size 6 \
--lr 1e-3 \ 
--num-class 19 \
--gpus 0 \
--factor 0.1 \
--csg 75 \
--apool \
--csg-stages 3.4 \
--chunks 8 \
--augment \
--evaluate \
--resume pretrained/csg_res101_segmentation_best.pth.tar \
  • Change --switch-model (deeplab50 or deeplab101) and --resume (path to pretrained checkpoints) accordingly.
  • Run CUDA_VISIBLE_DEVICES=0 bash train_seg.sh
    • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Train with CSG

  • Put the code below in train_seg.sh
python train_seg.py \
--epochs 50 \
--switch-model deeplab50 \
--batch-size 6 \
--lr 1e-3 \ 
--num-class 19 \
--gpus 0 \
--factor 0.1 \
--csg 75 \
--apool \
--csg-stages 3.4 \
--chunks 8 \
--augment
  • Change --switch-model (deeplab50 or deeplab101) accordingly.
  • Run CUDA_VISIBLE_DEVICES=0 bash train_seg.sh
    • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Citation

If you use this code for your research, please cite:

@article{chen2021contrastive,
  title={Contrastive syn-to-real generalization},
  author={Chen, Wuyang and Yu, Zhiding and Mello, SD and Liu, Sifei and Alvarez, Jose M and Wang, Zhangyang and Anandkumar, Anima},
  year={2021},
  publisher={ICLR}
}

csg's People

Contributors

chenwydj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

csg's Issues

Incomplete GTA5 to Cityscapes segmentation code and low mIoU

Hi,
Congratulations on your great work and thanks for releasing the code.

For the GTA5 to Cityscapes training code, I could not find the gta5_train.txt file. While I created it on my own, the code also does not map the GTA5 labels from 1-33 range to the Cityscapes label IDs (0-18) which leads to errors. I managed to fix that by performing the mapping. However, after training for around 4-5 epochs, the Cityscapes validation mIoU is very low (~0.5 to 1% only) and not increasing. I find this very odd since plain training on GTA5 usually quickly gives at least 15-25% mIoU. There is no model specified as initialization and I believe the PyTorch ImageNet-pretrained one is being used.

Kindly advise on whether some other initialization is required or some code additions are required or some preprocessing is required for the datasets. It would be very helpful if you could release the corrected code.

Question about the kde diagram.

Hi, thanks for your great work and for releasing the code.

I want to draw a Gaussian kernel density estimation diagram like figure 1, can you please release the code or give me some advice on how you plotted it?

Thanks for reading.

Reproducing paper results

Hello Mr. Chen!

thanks for the great work and repository!
Currently, I'm trying to reproduce your numbers stated in the paper. Using your provided code and checkpoints, I cannot achieve the same results.
In my case, I get an mIoU on Cityscapes if 36,141% on the ResNet50 checkpoint.
Does the provided checkpoint differ from the one used for generating the paper results?

Thanks.

Oracle model selection is used in this work?

Hi @chenwydj! Thanks for this outstanding work. Your public code really helps me a lot.

But I am a little confused about the model selection procedure, since the best model evaluated on the target domain (Cityscapes) is saved. This seems not a popular way to select models in domain generalization. Or is this the popular way in the current GTA5->Cityscapes task?

Pls let me know if I miss something. Thank you so much.

How to obtain 2-d feature for kde diagram?

Hi there,

Thanks much for sharing the code and great work!
Quick question on the features used for kde - which method was used to convert the high-dimensional features after glocal average pooling for the gaussian kde? Is this TSNE or some other method?

Thanks again!
YS

Question about the segmentation model on the paper.

Hi, Thanks for your great work!

I have a question about the segmentation model you used in your paper.
The paper on the segmentation model you referred to is as follows. https://arxiv.org/pdf/1706.05587.pdf
As far as I know, this paper is about Deeplabv3.
However, in the paper, it is marked as Deeplabv2.
Can you tell me what is the difference between the segmentation model you used and deeplabv3?

Thank you for reading.

E_s: hyperspherical energy of features

Hi Team,
Thank you for the great work and releasing the code.

We are wondering about the Es: hyperspherical energy of features part in the paper, however in the code base we couldn't find it.

It would be great, if you can guide us or have the implementation part of it to get the E_s part , similar to Table 1 (https://arxiv.org/pdf/2104.02290.pdf)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.