This repo is the official implementation of "Efficient generative adversarial networks using linear additive-attention Transformers".

By Emilio Morales-Juarez and Gibran Fuentes-Pineda.

Although the capacity of deep generative models for image generation, such as Diffusion Models (DMs) and Generative Adversarial Networks (GANs), has dramatically improved in recent years, much of their success can be attributed to computationally expensive architectures. This has limited their adoption and use to research laboratories and companies with large resources, while significantly raising the carbon footprint for training, fine-tuning, and inference. In this work, we present LadaGAN, an efficient generative adversarial network that is built upon a novel Transformer block named Ladaformer. The main component of this block is a linear additive-attention mechanism that computes a single attention vector per head instead of the quadratic dot-product attention. We employ Ladaformer in both the generator and discriminator, which reduces the computational complexity and overcomes the training instabilities often associated with Transformer GANs. LadaGAN consistently outperforms existing convolutional and Transformer GANs on benchmark datasets at different resolutions while being significantly more efficient. Moreover, LadaGAN shows competitive performance compared to state-of-the-art multi-step generative models (e.g. DMs) using orders of magnitude less computational resources.

• Python 3.9
• Tensorflow <= 2.13.1

A conda environment can be created and activated with:

conda create --name tf13 python=3.9.16
conda activate tf13
pip install tensorflow[and-cuda]==2.13.1 
pip install numpy matplotlib pillow scipy tqdm huggingface-hub

Use --file_pattern=<file_pattern> and --eval_dir=<eval_dir> to specify the dataset path and FID evaluation path.

python train.py --file_pattern=./data_path/*png --eval_dir=./eval_path/*png

Using a single 12GB GPU (RTX 3080 Ti) for training on CIFAR-10 and CelebA datasets takes less than 40 hours:

Adjust hyperparameters in the config.py file.

Implementation notes:

• This model depends on other files that may be licensed under different open source licenses.
• LadaGAN uses Differentiable Augmentation. Under BSD 2-Clause "Simplified" License.
• FID evaluation.
• Efficient patch generation with XLA.

Single head maps training progress:

@article{morales2024efficient,
  title={Efficient generative adversarial networks using linear additive-attention Transformers},
  author={Morales-Juarez, Emilio and Fuentes-Pineda, Gibran},
  journal={arXiv preprint arXiv:2401.09596},
  year={2024}
}

MIT