shalshamsi / abstention-ood Goto Github PK

This repository contains the part of supplementary Python code for our paper(preprint link) on robustness and distribution shift.

This method was able to achieve state-of-the-art performance on various Vision and Language Out-of-distribution detection tasks.

The code was tested with Python 3.7+ using PyTorch v1.2+. Additional Python libraries may be required, as specified in requirements.txt.

The pre-training and evaluation code makes use of the following datasets:

Training with background data further requires ILSVRC'12 or Tiny Images:

All datasets must be downloaded and prepared under paths specified in datasets/__init__.py. **Currently it is hardcoded to sushil's local device. ** ILSVRC'12 data should be further processed into LMDB format for faster data loading.

Use train.py to pretrain models on the in-distribution datasets. Models are automatically saved under checkpoints/ folder. Example:

python train.py --gpus 0 -a wrn40 -d cifar10 --epochs 100

Use cv_train_bg_resample.py to perform adversarial resampling on background dataset. Resampling weights are automatically saved under checkpoints/ folder. Example:

python cv_train_bg_resample.py --gpus 0 -a wrn40 -id cifar10 -od tiny_images --epochs 50 --load-path path/to/model.pth

Use train_bg.py to finetune models using background data. Supports full background dataset, uniformly subsampled background, or resampled background using learned weights. Example:

python train_bg.py --gpus 0 -a wrn40 -id cifar10 -od tiny_images --epochs 50 --load-path path/to/model.pth
python train_bg.py --gpus 0 -a wrn40 -id cifar10 -od tiny_images --epochs 50 --load-path path/to/model.pth --resample random -p 0.1
python train_bg.py --gpus 0 -a wrn40 -id cifar10 -od tiny_images --epochs 50 --load-path path/to/model.pth --resample path/to/resample/weights.pth -p 0.1

Use test_ood.py to evaluate OOD detection of trained models on one or more test sets. Example:

python test_ood.py --gpus 0 -a wrn40 -id cifar10 -od gaussian uniform textures lsun svhn places --out-ratio 0.2 --load-path path/to/model.pth

1. Background Data Resampling for Outlier-Aware Classification, Li, Yi and Vasconcelos, Nuno, CVPR 2020 | code