git clone https://github.com/subhc/unsup-parts.git
cd unsup-parts
conda env create --file environment.yml
conda activate unsup-parts
wget https://www.robots.ox.ac.uk/~vgg/research/unsup-parts/files/checkpoints.tar.gz
tar zxvf checkpoints.tar.gzThe project uses Weights & Biases for visualization, please update wandb_userid in train.py to your username
- Download CUB_200_2011.tgz and segmentations.tgz from the CUB-200-2011 provided links.
- Download cachedir.tar.gz mentioned here.
- Create a directory named
datawith the following folder structure inside and extract the tars at the mentioned locations. - Train a segmentation network to predict foreground masks for the test split, or download precalculated outputs: cub_supervisedlabels.tar.gz (17MB).
data
└── CUB # extract CUB_200_2011.tgz, cub_supervisedlabels.tar.gz here
├── CUB_200_2011 # extract cachedir.tar.gz and segmentations.tgz here
│ ├── attributes
│ ├── cachedir
│ ├── images
│ ├── parts
│ └── segmentations
└── supervisedlabels
Example
mkdir -p data/CUB/
cd data/CUB/
tar zxvf CUB_200_2011.tgz
tar zxvf cub_supervised_labels.tar.gz
cd CUB_200_2011
tar zxvf segmentations.tgz
tar zxvf cachedir.tar.gz
- Create a directory named
datawith the folder structure below. - Download the segmentation folder from the DeepFashion provided links.
- Extract img_highres_seg.zip inside
segmentationFolder. - Train a segmentation network to predict foreground masks for the test split, or download precalculated outputs: deepfashion_supervisedlabels.tar.gz (56MB).
data
└── DeepFashion
└── In-shop Clothes Retrieval Benchmark # extract deepfashion_supervisedlabels.tar.gz here
├── Anno
│ └── segmentation # extract img_highres_seg.zip here
│ └── img_highres
│ ├── MEN
│ └── WOMEN
└── supervisedlabels
└── img_highres
├── MEN
└── WOMEN
Example
mkdir -p data/DeepFashion/In-shop\ Clothes\ Retrieval\ Benchmark/Anno/
cd data/DeepFashion/In-shop\ Clothes\ Retrieval\ Benchmark/
wget https://www.robots.ox.ac.uk/~vgg/research/unsup-parts/files/deepfashion_supervisedlabels.tar.gz
tar zxvf deepfashion_supervisedlabels.tar.gz
cd Anno
# get the segmentation folder from the google drive link
cd segmentation
unzip img_highres_seg.zipTo train CUB:
python train.py dataset_name=CUBTo train DeepFashion:
python train.py dataset_name=DFYou can find evaluation code in the evaluation folder.
| Description | Size | Link |
|---|---|---|
| CUB-200-2011 (pth) | 181MB | here |
| DeepFashion (pth) | 181MB | here |
| Both (tar.gz) | 351MB | here |
Please move the pth files in the checkpoints/CUB and checkpoints/DeepFashion folders respectively.
The goal of self-supervised visual representation learning is to learn strong, transferable image representations, with the majority of research focusing on object or scene level. On the other hand, representation learning at part level has received significantly less attention. In this paper, we propose an unsupervised approach to object part discovery and segmentation and make three contributions. First, we construct a proxy task through a set of objectives that encourages the model to learn a meaningful decomposition of the image into its parts. Secondly, prior work argues for reconstructing or clustering pre-computed features as a proxy to parts; we show empirically that this alone is unlikely to find meaningful parts; mainly because of their low resolution and the tendency of classification networks to spatially smear out information. We suggest that image reconstruction at the level of pixels can alleviate this problem, acting as a complementary cue. Lastly, we show that the standard evaluation based on keypoint regression does not correlate well with segmentation quality and thus introduce different metrics, NMI and ARI, that better characterize the decomposition of objects into parts. Our method yields semantic parts which are consistent across fine-grained but visually distinct categories, outperforming the state of the art on three benchmark datasets. Code is available at the project page.
@inproceedings{choudhury21unsupervised,
author = {Subhabrata Choudhury and Iro Laina and Christian Rupprecht and Andrea Vedaldi},
booktitle = {Proceedings of Advances in Neural Information Processing Systems (NeurIPS)},
title = {Unsupervised Part Discovery from Contrastive Reconstruction},
year = {2021}
}
Code is largely based on SCOPS.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent