This is the implementation of the IcGAN model proposed in our paper:

Invertible Conditional GANs for image editing. November 2016.

This paper is a summarized and updated version of my master thesis, which you can find here:

Master thesis: Invertible Conditional Generative Adversarial Networks. September 2016.

The baseline used is the Torch implementation of the DCGAN by Radford et al.

1. Training the model
   1. Face dataset: CelebA
   2. Digit dataset: MNIST
2. Visualize the results
   1. Reconstruct and modify real images
   2. Swap attributes
   3. Interpolate between faces

Please refer to DCGAN torch repository to know the requirements and dependencies to run the code. Additionally, you will need to install the threads package:

luarocks install threads

In order to interactively display the results, follow these steps.

The IcGAN is trained in four steps.

1. Train the generator.
2. Create a dataset of generated images with the generator.
3. Train the encoder Z to map an image x to a latent representation z with the dataset generated images.
4. Train the encoder Y to map an image x to a conditional information vector y with the dataset of real images.

All the parameters of the training phase are located in cfg/mainConfig.lua.

There is already a pre-trained model for CelebA available in case you want to skip the training part. Here you can find instructions on how to use it.

Note: for speed purposes, the whole dataset will be loaded into RAM during training time, which requires about 10 GB of RAM. Therefore, 12 GB of RAM is a minimum requirement. Also, the dataset will be stored as a tensor to load it faster, make sure that you have around 25 GB of free space.

mkdir celebA; cd celebA

Download img_align_celeba.zip here under the link "Align&Cropped Images".

unzip img_align_celeba.zip; cd ..
DATA_ROOT=celebA th data/preprocess_celebA.lua

• Conditional GAN: parameters are already configured to run CelebA (dataset=celebA, dataRoot=celebA).

   th trainGAN.lua

• Generate encoder dataset:

   net=[GENERATOR_PATH] outputFolder=celebA/genDataset samples=182638 th data/generateEncoderDataset.lua

  (GENERATOR_PATH example: checkpoints/celebA_25_net_G.t7)

• Train encoder Z:

   datasetPath=celebA/genDataset/ type=Z th trainEncoder.lua
  

• Train encoder Y:

   datasetPath=celebA/ type=Y th trainEncoder.lua
  

Download MNIST as a luarocks package: luarocks install mnist

• Conditional GAN:

   name=mnist dataset=mnist dataRoot=mnist th trainGAN.lua

• Generate encoder dataset:

   net=[GENERATOR_PATH] outputFolder=mnist/genDataset samples=60000 th data/generateEncoderDataset.lua

  (GENERATOR_PATH example: checkpoints/mnist_25_net_G.t7)

• Train encoder Z:

   datasetPath=mnist/genDataset/ type=Z th trainEncoder.lua
  

• Train encoder Y:

   datasetPath=mnist type=Y th trainEncoder.lua
  

CelebA model is available for download here. The file includes the generator and both encoders (encoder Z and encoder Y).

For visualizing the results you will need an already trained IcGAN (i.e. a generator and two encoders). The parameters for generating results are in cfg/generateConfig.lua.

decNet=celeba_24_G.t7 encZnet=celeba_encZ_7.t7 encYnet=celeba_encY_5.t7 loadPath=[PATH_TO_REAL_IMAGES] th generation/reconstructWithVariations.lua

Swap the attribute information between two pairs of faces.

decNet=celeba_24_G.t7 encZnet=celeba_encZ_7.t7 encYnet=celeba_encY_5.t7 im1Path=[IM1] im2Path=[IM2] th generation/attributeTransfer.lua

decNet=celeba_24_G.t7 encZnet=celeba_encZ_7.t7 encYnet=celeba_encY_5.t7 im1Path=[IM1] im2Path=[IM2] th generation/interpolate.lua

Do you like or use our work? Please cite us as

@article{Perarnau2016,
  author    = {Guim Perarnau and
               Joost van de Weijer and
               Bogdan Raducanu and
               Jose M. \'Alvarez},
  title     = {{Invertible Conditional GANs for image editing}},
  journal   = {NIPS Workshop on Adversarial Training},
  year      = {2016},
}