Lack of Knowledge Database Support in Vision Mode for Bedrock Claude 3 about aws-genai-llm-chatbot HOT 2 OPEN

Thank you for spotlighting the absence of knowledge database support in Bedrock Claude 3's Vision mode. I think many will want this!

The LangChain blog post suggests three approaches for implementing multi-modal RAG:

1. Using multi-modal embeddings (e.g., Amazon Titan Multimodal Embeddings model) to generate multimodal (image & text) embeddings and search those embeddings
2. Using a multi-modal LLM (Claude 3 Vision) to generate image text summaries, then searching those summaries with text/table content
3. Combining (2) with raw image retrieval, allowing a multi-modal LLM to incorporate images directly in responses

Which approach best fits your Claude's Vision mode use case? Did you want option 1, 2, 3, or another variant?

from aws-genai-llm-chatbot.

I'm handle the case for IT support system when users uploading image and asking how to fix the issue. The solution 3 will be the best choice at this time due to complexity and cost of multi-modal embedding.

But, for retrieval, I will generate semantic query using LLM from histories + user's question + image.

Thanks for advice.

from aws-genai-llm-chatbot.