FID score of VAE with RealNVP prior about srvae HOT 3 CLOSED

Hi!

First of all, thank you for your interest for our work!

FID, unlike likelihood (in this kind of generative models we stick mostly to bpd), can be a very ambiguous score, a little twist to the data, number of comparing data and the inception model itself can alter the final score. In out case, our score came from the process that it is described in the paper; 50k generated images and 50k real images from the train set, saved with the torchvision's save_image function (this also matters). We verified the scores with the inception weights of tensorflow's and pytorch's (copied from tf) version.

from srvae.

Thanks for the explanation!

I try save_image by adding the following lines

for i in range(n_samples): 
    save_image(x[i], save_path%i)

at

Then generate 50k images (in png) and evaluate with the tensorflows version here https://github.com/bioinf-jku/TTUR with their released precalculated statistics of cifar10's training set. And it gives FID: 58.67528283730496. The pytorch version gives 58.38 with the same input. But the FID score on training set in the paper (table1, row1, score in the brackets) is 37.25.

Do you idea of which step I am doing wrong? Or it would be really appreciated if you could share the snippets you used to generate the images to help me debug with the code!

from srvae.

In our version, we also saved the CIFAR images with torch to a folder and did folder comparison.
I don't believe that you have any bug on your code

from srvae.