Code Monkey home page Code Monkey logo

jdarke / real-time-latent-consistency-model Goto Github PK

View Code? Open in Web Editor NEW

This project forked from radames/real-time-latent-consistency-model

0.0 0.0 0.0 190 KB

Demo showcasing ~real-time Latent Consistency Model pipeline with Diffusers and a MJPEG stream server

Home Page: https://huggingface.co/spaces/radames/Real-Time-Latent-Consistency-Model

License: Apache License 2.0

Shell 0.50% JavaScript 1.41% Python 62.18% TypeScript 11.48% CSS 0.07% HTML 0.41% Dockerfile 1.71% Svelte 22.25%

real-time-latent-consistency-model's Introduction

title emoji colorFrom colorTo sdk pinned suggested_hardware
Real-Time Latent Consistency Model Image-to-Image ControlNet
๐Ÿ–ผ๏ธ๐Ÿ–ผ๏ธ
gray
indigo
docker
false
a10g-small

Real-Time Latent Consistency Model

This demo showcases Latent Consistency Model (LCM) using Diffusers with a MJPEG stream server. You can read more about LCM + LoRAs with diffusers here.

You need a webcam to run this demo. ๐Ÿค—

See a collecting with live demos here

Running Locally

You need CUDA and Python 3.10, Node > 19, Mac with an M1/M2/M3 chip or Intel Arc GPU

Install

python -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt
cd frontend && npm install && npm run build && cd ..
python run.py --reload --pipeline controlnet

Pipelines

You can build your own pipeline following examples here here, don't forget to fuild the frontend first

cd frontend && npm install && npm run build && cd ..

LCM

Image to Image

python run.py --reload --pipeline img2img 

LCM

Text to Image

python run.py --reload --pipeline txt2img 

Image to Image ControlNet Canny

python run.py --reload --pipeline controlnet 

LCM + LoRa

Using LCM-LoRA, giving it the super power of doing inference in as little as 4 steps. Learn more here or technical report

Image to Image ControlNet Canny LoRa

python run.py --reload --pipeline controlnetLoraSD15

or SDXL, note that SDXL is slower than SD15 since the inference runs on 1024x1024 images

python run.py --reload --pipeline controlnetLoraSDXL

Text to Image

python run.py --reload --pipeline txt2imgLora

or

python run.py --reload --pipeline txt2imgLoraSDXL

Setting environment variables

TIMEOUT: limit user session timeout
SAFETY_CHECKER: disabled if you want NSFW filter off
MAX_QUEUE_SIZE: limit number of users on current app instance
TORCH_COMPILE: enable if you want to use torch compile for faster inference works well on A100 GPUs USE_TAESD: enable if you want to use Autoencoder Tiny

If you run using bash build-run.sh you can set PIPELINE variables to choose the pipeline you want to run

PIPELINE=txt2imgLoraSDXL bash build-run.sh

and setting environment variables

TIMEOUT=120 SAFETY_CHECKER=True MAX_QUEUE_SIZE=4 python run.py --reload --pipeline txt2imgLoraSDXL

If you're running locally and want to test it on Mobile Safari, the webserver needs to be served over HTTPS, or follow this instruction on my comment

openssl req -newkey rsa:4096 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem
python run.py --reload --ssl-certfile=certificate.pem --ssl-keyfile=key.pem

Docker

You need NVIDIA Container Toolkit for Docker, defaults to `controlnet``

docker build -t lcm-live .
docker run -ti -p 7860:7860 --gpus all lcm-live

or with environment variables

docker run -ti -e PIPELINE=txt2imgLoraSDXL -p 7860:7860 --gpus all lcm-live

Development Mode

python run.py --reload  

Demo on Hugging Face

https://huggingface.co/spaces/radames/Real-Time-Latent-Consistency-Model

lcm-real.mp4

real-time-latent-consistency-model's People

Contributors

radames avatar nuullll avatar cocktailpeanut avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.