cd cvpods
ln -s /path/to/your/ImageNet datasets/imagenet
Train your own models
cd /path/to/your/SelfSup/examples/simclr/simclr.res50.scratch.imagenet.224size.256bs.200e
# pre-train
pods_train --num-gpus 8
# convert to weights
python convert.py simclr.res50.scratch.imagenet.224size.256bs.200e/log/model_final.pth weights.pkl
# downstream evaluation
cd /path/to/your/simclr.res50.scratch.imagenet.224size.256bs.200e.lin_cls
pods_train --num-gpus 8 MODEL.WEIGHTS /path/to/your/weights.pkl
Model Zoo
Supervised Classification
ImageNet
Methods
Training Schedule
Top 1 Acc
Res50
100e
76.4
CIFAR 10
Methods
Training Schedule
Top 1 Acc
Res50
200e
95.4
STL 10
Methods
Training Schedule
Top 1 Acc
Res50
150e
86.1
Self-Supervised Learning - Classification
All results in the below table are trained using resnet-50 and reported on the ILSVRC2012 dataset.
Methods
Training Schedule
Batch Size
Our Acc@1
Official Acc@1
MoCo
200e
256
60.5
60.5
MoCov2
200e
256
67.6
67.5
SimCLR
200e
256
63.2
61.9
SimCLR*
200e
256
67.3
Ours
SiMo
200e
256
68.1
Ours
SimSiam
100e
256
67.6
67.7
SwAV
200e
256
73.0
72.7
BYOL
100e
2048
69.8
66.5 (bs4096 from SimSiam paper)
BarlowTwins
300e
1024
Comming Soon
71.7
Self-Supervised Learning - Detection (2D)
All the results reported below are trained on ILSVRC2012 and evaluated on MS COCO using Faster-RCNN-FPN and resnet-50.
Methods
Training Schedule
Batch Size
Box AP
SCRL
200
4096
39.9 ( official: 40.5 with bs 8192)
DetCon
200
256
Comming Soon.
Self-Supervised Learning - 3D Scene Understanding
Methods
Training Schedule
Downstream task
PointContrast
-
Comming Soon.
Citation
SelfSup is a part of cvpods, so if you find this repo useful in your research, or if you want to refer the implementations in this repo, please consider cite:
@article{zhu2020eqco,
title={EqCo: Equivalent Rules for Self-supervised Contrastive Learning},
author={Zhu, Benjin and Huang, Junqiang and Li, Zeming and Zhang, Xiangyu and Sun, Jian},
journal={arXiv preprint arXiv:2010.01929},
year={2020}
}
@misc{zhu2020cvpods,
title={cvpods: All-in-one Toolbox for Computer Vision Research},
author={Zhu*, Benjin and Wang*, Feng and Wang, Jianfeng and Yang, Siwei and Chen, Jianhu and Li, Zeming},
year={2020}
}
Hi,
I am really new to self supervised learning. I would like to know how do you adjust no.of epochs when you train on multi-gpu.
I am intending to train on ImageNet. Unfortunately, I cannot fit mini-batch of 256 in one gpu. As for as I understand, when I train on 4-gpu, even if I train for 100 epochs, the no.of epochs trained will be effectively 100/4=25 as number of gradient updates will also be divided by 4. Please correct me if I am wrong.
Please let me know how have you accounted for that in your implementation.