donahowe / theatergen Goto Github PK

We propose Theatergen, a tuning-free method for consistent multi-turn image generation. The key idea is to utilize LLM for character management with layout and id and customize each character to avoid attention leakage. We further propose the CMIGBench for evaluating the consistency in multi-turn image generation.

• Deployment with GPT interface
• Release Benchmark
• Release code

• [2024.04.26] We have released our code and benchmark

To install requirements:

pip install -r requirements.txt

Generate with CMIGBench or replace with your own demo

python generate.py --task story --sd_version '1.5' --dataset_path CMIGBench

Prepare the output in the following format

├── output_dir
|   ├── dialogue 1
|      ├── turn1.png 
|      ├── turn2.png 
|      ├── turn3.png 
|      └── turn4.png 
|   ├── dialogue 2
|      ...

Evalutate the generated results of CMIGBench

python CMIGBench/eval/eval.py 
python CMIGBench/eval/eval_extra.py 

If you have any questions, please feel free to email us at [email protected].

🌟🌟🌟(I am an undergraduate student actively seeking opportunities for a Ph.D. program in 25 fall.)🌟🌟🌟

Our work is based on stable diffusion, Grounded-SAM, T2I-Adapter, and IP-Adapter. We appreciate their outstanding contributions.

If you found this code helpful, please consider citing:

@article{cheng2024theatergen,
  title={TheaterGen: Character Management with LLM for Consistent Multi-turn Image Generation},
  author={Cheng, Junhao and Yin, Baiqiao and Cai, Kaixin and Huang, Minbin and Li, Hanhui and He, Yuxin and Lu, Xi and Li, Yue and Li, Yifei and Cheng, Yuhao and others},
  journal={arXiv preprint arXiv:2404.18919},
  year={2024}
}