StableDiffusion from scratch (pytorch lightning)
├── StableDiffusion
├── data
├── Attention.py
├── CLIP.py
├── DDPM.py
├── Decoder.py
├── Diffusion.py
├── Encoder.py
├── Inference.py
├── ModelConverter.py
├── ModelLoader.py
├── Pipeline.py
├── requirments.txt
└── README.md
data
: Model Weights, CLIP ToknizerDDPM.py
: DDPM SamplerEncoder/Decoder
: VAE Encoder/DecoderModelConverter.py
: load from standard weightsModelLoader.py
: Construct StableDiffusiondiffusion.py
: Diffusion Forward/Backward ProcessPipeline.py
: Generating Pipeline
Download vocab.json and merges.txt from https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main/tokenizer and save them in the data folder
Download v1-5-pruned-emaonly.ckpt from https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main and save it in the data folder
# Clone this repo and install dependency
git clone https://github.com/inhopp/StableDiffusion.git
pip install -r "StableDiffusion/requirments.txt"
# Text to Image
prompt = "a dog, fullmetal, robotic, mechanical parts, highly detailed, ultra sharp, cinematic, 100mm lens, 8k resolution."
uncond_prompt = "" # Also known as negative prompt
do_cfg = True
cfg_scale = 8 # min: 1, max: 14
# Image to Image
input_image = None
image_path = "./images/dog.jpg"
strength = 0.9
sampler = "ddpm"
num_inference_steps = 50
seed = 42