A repository to consolidate stable diffusion finetuning scripts in to a training hub. Train inpainting, depth, v1+, v2+, image variations, image colorization, whatever. Train with optimizations like 8-bit adam and xformers for faster and more memory efficient training.

• Train depth
• Train inpaint
• Train on custom image input (image latent concat to noise latent) *idea from Justin Pinkey
• Train on custom conditionings (image embeddings instead of text for example) *idea from Justin Pinkey
• Use filenames as prompts
• Use bnb 8-bit adam for more memory efficient training
• Use xformers for more memory efficient training
• Mixed precision (fp16/bf16) training
• Prompt shuffling (split on ',' or a custom string and shuffle based on a given probability)
• Train depth with custom depth images

Pull requests, discussions, requests, suggestions, and critiques are all welcome! :)

This is a combination of a bunch of repos as well as my own code and edits on scripts. I will do my best to give credit where credit is due in the form of comments, licenses, a shout-out on the readme, etc. If I happen to miss giving anyone credit/include a license please email me at [email protected] and I will fix it!

• Huge thanks to Hugging Face for the diffusers library that makes most of this code possible
• Huge thanks to Stable Diffusion for creating the actual diffusion model and open sourcing it
• Thanks to epitaque for depth training
• Another thanks to Hugging Face for inpainting training
• Shoutout to EveryDream for windows venv setup and bnb patch
• Shoutout to Justin Pinkey/Lambda Labs for research in to training with different inputs

Reach out to [email protected] with any requests/questions/comments/suggestions/concerns

If you're interested in training text-to-audio latent diffusion go check out https://github.com/serp-co/ai-text-to-audio-latent-diffusion