Leigang Qu, Haochuan Li, Tan Wang*, Wenjie Wang*, Yongqi Li, Liqiang Nie, and Tat-Seng Chua
(* Corresponding Authors)
National University of Singapore, Nanyang Technological University, Hong Kong Polytechnic University, Harbin Institute of Technology (Shenzhen)
This repository contains code and links to the Unified Text-to-Image Generation and Retrieval (TIGeR) work. We show the potential of intrinsic discriminative abilities of current Multimodal Large Language Models (MLLMs) and propose a training-free method to unify text-to-image generation and retrieval. Besides, we build a benchmark called TIGeR-Bench to comprehensively evaluate the unified performance across recent MLLMs.
Overview of the framework to unify text-to-image generation and retrieval. Images from the database are first tokenized into discrete codes and a lookup table is maintained for the correspondence between discrete codes and images. The given prompt X is first fed into a MLLM and Forward Beam Search is performed to retrieve and generate images in parallel. The prompt and obtained images are then fed into the same MLLM for Reverse Re-Ranking and Decision-making.
- Release the evaluation code for the unified task and the retrieval task.
- Release the inference code for unified text-to-image generation and retrieval.
- 2024-7-6 Release TIGeR-Bench on Huggingface.
- 2024-6-9 Release the paper of TIGeR on arXiv.
We thank the authors of SEED-LLaMA and LaVIT for making their code available.
If you find our work useful in your research, please consider citing TIGeR:
@article{qu2024unified,
title={Unified Text-to-Image Generation and Retrieval},
author={Qu, Leigang and Li, Haochuan and Wang, Tan and Wang, Wenjie and Li, Yongqi and Nie, Liqiang and Chua, Tat-Seng},
journal={arXiv preprint arXiv:2406.05814},
year={2024}
}