stablediffusion's Introduction

StableDiffusion

StableDiffusion from scratch (pytorch lightning)

{: width="50%" height="60%"}

Repository Directory

├── StableDiffusion
        ├── data
        ├── Attention.py
        ├── CLIP.py
        ├── DDPM.py
        ├── Decoder.py
        ├── Diffusion.py
        ├── Encoder.py
        ├── Inference.py
        ├── ModelConverter.py
        ├── ModelLoader.py
        ├── Pipeline.py
        ├── requirments.txt
        └── README.md

data : Model Weights, CLIP Toknizer
DDPM.py : DDPM Sampler
Encoder/Decoder : VAE Encoder/Decoder
ModelConverter.py : load from standard weights
ModelLoader.py : Construct StableDiffusion
diffusion.py : Diffusion Forward/Backward Process
Pipeline.py : Generating Pipeline

Tutoral

Clone repo and install model and depenency

Download vocab.json and merges.txt from https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main/tokenizer and save them in the data folder
Download v1-5-pruned-emaonly.ckpt from https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main and save it in the data folder

# Clone this repo and install dependency
git clone https://github.com/inhopp/StableDiffusion.git
pip install -r "StableDiffusion/requirments.txt"

Inference.py

# Text to Image
prompt = "a dog, fullmetal, robotic, mechanical parts, highly detailed, ultra sharp, cinematic, 100mm lens, 8k resolution."
uncond_prompt = ""  # Also known as negative prompt
do_cfg = True
cfg_scale = 8  # min: 1, max: 14

# Image to Image
input_image = None
image_path = "./images/dog.jpg"
strength = 0.9

sampler = "ddpm"
num_inference_steps = 50
seed = 42