Reproducing accuracy about uncertainty-baselines HOT 5 CLOSED

Hey @Johswald! Thanks for asking. Have you verified that you're running under 8 accelerators? That's the exact setting in the default flags, which reproduces the result (we've checked on cloud TPUs). This is often the culprit, and may suggest we should somehow raise an error if, e.g., the system's # accelerators doesn't match FLAGS.num_cores.

from uncertainty-baselines.

Hey @dustinvtran
No, I am just running it on a single GPU which is also quite fast. I can try to parallelise on my node with 8 GPUs if you think this would make a difference? Algorithmically I should not right? (No experience with TPUs :) )

I was asking about the config because it was not clear if this is a cifar10 or cifar100 config. So the one that is the default one in batchensemble.py is the cifar100 one that gets the 81.9% test set (on TPUs)? Are you using the same config for cifar10 and 100?
Thank you again!

from uncertainty-baselines.

All the CIFAR uses num_cores=8 currently, which means 8 GPUs. This affects the global batch size for each gradient step. The default flag values are for cifar10 and I think you need to just change FLAGS.dataset for cifar100.

from uncertainty-baselines.

Ah of course, thanks a lot! I will get back to you once the runs are done.

from uncertainty-baselines.

Closing due to inactivity, please reopen if needed!

from uncertainty-baselines.