An AI tool that receives an image and uses generative AI to create a narrated short story in an audio file about the context of the image.

The tool utilizes Hugging Face Transformers open-source framework for deep learning in combination with OpenAI prompting via Langchain framework.

Execution is devided into 3 parts:

• Image to text: an image-to-text transformer model (Salesforce/blip-image-captioning-base) is used to generate a text scenario based on the on the AI understanding of the image context
• Text to story: OpenAI LLM model is prompted to create a short story based on the generated scenario
• Story to speech: a text-to-speech transformer model (espnet/kan-bayashi_ljspeech_vits) is used to convert the generated short story into a voice-narrated audio file
• A user interface is built using streamlit to enable uploading the image and playing the audio file

• os
• dotenv
• transformers
• langchain
• requests
• streamlit

• Before using the app, the user should have personal tokens for Hugging Face and Open AI
• The user should save the personal tokens in an ".env" file within the package as string objects under object names: HUGGINGFACE_TOKEN and OPENAI_TOKEN
• The user can then run the app using the command: streamlit run app.py
• Once the app is running on streamlit, the user can upload the target image
• Execution will start automatically and it may take a few minutes to complete
• Once completed, the app will display:
  • The scenario text generated by the image-to-text transformer model
  • The short story generated by prompting the OpenAI LLM
  • The autio file narrating the short story generated by the text-to-speech transformer model
• The audio file along with the image will be saved in "audio-img" folder inside the package