AI generate video, image and audio from text prompts or strips.

• Text to video
• Text to audio
• Text to image
• Seed, Quality Steps, Frames, Word Power.
• Batch operation of text strips into videos, audio, music, speech or images.
• Model card selector
• Render finished notification
• Model Cards: Modelscope, Animov, Stable Diffusion 1.5 & 2, Deep Floyd IF, AudioLMD and Bark
• One click install and uninstall dependencies
• User defined file path for generated files
• Seed added to strip name

• A CUDA supported Nvidia card with at least 4 GB VRam.

(As for Linux and MacOS, if anything differs in installation, then please share instructions.)

• First you must download and install git for your platform(must be on PATH(or Bark will fail)): https://git-scm.com/downloads

• Download the add-on: https://github.com/tin2tin/text_to_video/archive/refs/heads/main.zip

• On Windows, right click on the Blender icon and "Run Blender as Administrator"(or you'll get write permission errors).

• Install the add-on as usual: Preferences > Add-ons > Install > select file > enable the add-on.

• In the Generative AI add-on preferences, hit the "Install all Dependencies" button.

• Note that you can change what model cards are used in the various modes here(video, image, audio).

• When it writes that it is finished(if any errors, let me know).

• Open the add-on UI in the Sequencedr > Sidebar > Generative AI.

• The first time any model is executed many GB will have to be downloaded, so go grab lots of coffee.

• If it says:"ModuleNotFoundError: Refer to https://github.com/facebookresearch/xformers for more information on how to install xformers", then try to restart Blender.

Install Dependencies, set Movie Model Card and set Sound Notification in the add-on preferences:

Video Sequence Editor > Sidebar > Generative AI

Converting Text strips into GeneratorAI strips:

The Animov models have been trained on Anime material, so adding "anime" to the prompt is necessary, especially for the Animov-512x model. The Stable Diffusion models for generating images have been used a lot, so there are plenty of prompt suggestions out there if you google for them. The Modelscope model has a watermark, since it's been trained on Shutterstock material, and can't be used for commercial purposes.

Accessed through the Add menu > Text to GenerativeAI

Find AudioLDM documentation here: https://github.com/haoheliu/AudioLDM Try prompts like: Bag pipes playing a funeral dirge, punk rock band playing hardcore song, techno dj playing deep bass house music, and acid house loop with jazz. Or: Voice of God judging mankind, woman talking about celestial beings, hammer on wood.

Find Bark documentation here: https://github.com/suno-ai/bark

• [laughter]
• [laughs]
• [sighs]
• [music]
• [gasps]
• [clears throat]
• — or ... for hesitations
• ♪ for song lyrics
• capitalization for emphasis of a word
• MAN/WOMAN: for bias towards speaker

Speaker Library: https://suno-ai.notion.site/8b8e8749ed514b0cbf3f699013548683?v=bc67cff786b04b50b3ceb756fd05f68c

Diffusers: https://github.com/huggingface/diffusers

ModelScope: https://modelscope.cn/models/damo/text-to-video-synthesis/summary

Animov: https://huggingface.co/vdo/animov-0.1.1

Potat1: https://huggingface.co/camenduru/potat1

Zeroscope Dark: https://huggingface.co/cerspense/zeroscope_v2_dark_30x448x256

Zeroscope XL: https://huggingface.co/cerspense/zeroscope_v2_XL

Polyware's T2V: https://huggingface.co/polyware-ai/text-to-video-ms-stable-v1

Longscope: https://huggingface.co/polyware-ai/longscope

AudioLDM: https://huggingface.co/cvssp/audioldm-s-full-v2 https://github.com/haoheliu/AudioLDM

Bark: https://github.com/suno-ai/bark

Deep Floyd IF: https://github.com/deep-floyd/IF

Stable Diffusion XL: https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9

Hugging Face diffusers model are downloaded from the hub and saved to a local cache directory. By default, the cache directory is located at:

On Linux and macOS: ~/.cache/huggingface/transformers

On Windows: %userprofile%.cache\huggingface\transformers

Here you can locate and delete the individual models.

• The models can only be used for non-commercial purposes. The models are meant for research purposes.
• The models was not trained to realistically represent people or events, so using it to generate such content is beyond the model's capabilities.
• It is prohibited to generate content that is demeaning or harmful to people or their environment, culture, religion, etc.
• Prohibited for pornographic, violent and bloody content generation.
• Prohibited for error and false information generation.