Reproducing OOD scores for CIFAR-10 vs SVHN about uncertainty-baselines HOT 5 OPEN

Hi,
can you please point me to the script that reproduces the OOD results?
Thanks

from uncertainty-baselines.

@mdabbah do you mean the script in this repository here or the code I used to reproduce the results?

from uncertainty-baselines.

the script in this repository

from uncertainty-baselines.

Hi Christoph!

So sorry for the late reply. Just want to make sure, did you treat OOD as positive label or negative label? (asking since AUPR is sensitive to label imbalance), and do you mind sharing the AUPR value in both case (i.e., the AUPR for the case OOD is treated as positive label, and the case that OOD is treated as negative label).

Thanks!
Jeremiah

from uncertainty-baselines.

Hello,

I was trying to reproduce the results from paper for deterministic WideResNet and SNGP WideResNet model trained on cifar10 and SVHN and CIFAR100 as OOD set.
I have attached my results below.

command used:
python3 baselines/cifar/deterministic.py --dataset=cifar10 --num_cores=2 --data_dir=./tensorflow_datasets --output_dir=./model_detr --use_gpu=true --download_data=True
python3 baselines/cifar/sngp.py --dataset=cifar10 --num_cores=2 --data_dir=./tensorflow_datasets --output_dir=./model --use_gpu=true --download_data=True
The deterministic OOD scores seem very different from paper. Can you help us see why that could be?

Thanks

from uncertainty-baselines.