Related work about caption-anything HOT 2 OPEN

@LengZhuo0831 As far as I know, dense captioning is the most related topic, which generates captions at the region/object level. Scene graph generation is another way to describe the image at the instance level, which considers the instance as graph nodes and relationships as edges.

Here I list several early seminal works

• image-based: DenseCap: Fully Convolutional Localization Networks for Dense Captioning
• video-based: Dense-Captioning Events in Videos
• 3D data-based: Scan2cap: Context-aware dense captioning in rgb-d scans

Some recent works to combine LLMs and fine-grained visual experts for dense captioning generation:

• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
• Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning
• ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions

from caption-anything.

One more to add for the LLMs + Image Captioners:

IC3: Image Captioning by Committee Consensus
https://arxiv.org/abs/2302.01328

from caption-anything.