qy-h00 / attention-interpolation-diffusion Goto Github PK

He Qiyuan¹,  Wang Jinghao²,  Liu Ziwei²,  Angela Yao^1,✉;
Computer Vision & Machine Learning Group, National University of Singapore ¹
S-Lab, Nanyang Technological University ^2
✉ Corresponding Author

[03/2024] Code and paper are publicly available.

TL;DR: AID (Attention Interpolation via Diffusion) is a training-free method that enables the text-to-image diffusion model to generate interpolation between different conditions with high consistency, smoothness and fidelity. Its variant, PAID, provides further control of the interpolation via prompt guidance.

Directly try PAID with Stable Diffusion 2.1 or SDXL using Google's Free GPU!

1. Clone the repository and install the requirements:

git clone https://github.com/QY-H00/attention-interpolation-diffusion.git
cd attention-interpolation-diffusion
pip install requirements.txt

2. Go to play.ipynb or play_sdxl.ipynb for fun!

1. install Gradio

pip install gradio

2. Launch the Gradio interface

gradio gradio_src/app.py

Our method offers users customized and diverse configurations to experiment with, allowing them to freely adjust settings and achieve a wide range of interesting interpolation results. Here are some examples:

If you found this repository/our paper useful, please consider citing:

@misc{he2024aid,
      title={AID: Attention Interpolation of Text-to-Image Diffusion}, 
      author={Qiyuan He and Jinghao Wang and Ziwei Liu and Angela Yao},
      year={2024},
      eprint={2403.17924},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

We thank the following repositories for their great work: diffusers, transformers.

Pikachu -> Gundam

Computer -> Phone

Ninja -> Cat

Ninja -> Dog

Starry night -> Mona Lisas

SkyCraper -> Town