Code Monkey home page Code Monkey logo

idm-vton's People

Contributors

subin-kim-cv avatar yisol avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

idm-vton's Issues

The following packages are not available from current channels

C:\tut\IDM-VTON>conda env create -f environment.yaml
Retrieving notices: ...working... done
Channels:
 - pytorch
 - nvidia
 - defaults
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - torchtriton==2.0.0=py310
  - pytorch-cuda==11.8=h7e8668a_5
  - pytorch==2.0.1=py3.10_cuda11.8_cudnn8.7.0_0
  - python==3.10.0=h12debd9_5
  - pip==23.3.1=py310h06a4308_0

Current channels:

  - https://conda.anaconda.org/pytorch/win-64
  - https://conda.anaconda.org/nvidia/win-64
  - https://repo.anaconda.com/pkgs/main/win-64
  - https://repo.anaconda.com/pkgs/r/win-64
  - https://repo.anaconda.com/pkgs/msys2/win-64

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

IDM-VTON is State of the Art Congrats - 1 Click Install and Use IDM-VTON on Windows (8GB VRAM), Massed Compute, RunPod and a Free Kaggle account Notebook full Tutorials

The IDM-VTON (Improving Diffusion Models for Authentic Virtual Try-on in the Wild) is so powerful that it can even transfer beard or hair as well.

I have prepared installer scripts and full tutorials for Windows (requires min 8 GB VRAM GPU), Massed Compute (I suggest this if you don’t have a strong GPU), RunPod and a free Kaggle account (works perfect as well but slow).

All the scripts with instructions can be downloaded from here : https://www.patreon.com/posts/virtual-try-on-1-103022942

image

Please use below link to register Massed Compute:

https://vm.massedcompute.com/signup?linkId=lp_034338&sourceId=secourses&tenantId=massed-compute

Use our Massed Compute coupon on A6000 to get half price : SECourses

Tutorials are as below

IDM-VTON: The Most Amazing Virtual Clothing Try On Application — Open Source — 1 Click Install & Use : https://youtu.be/m4pcIeAVQD0

image

IDM-VTON: The Most Amazing Virtual Clothing Try On Application — RunPod — Massed Compute — Kaggle : https://youtu.be/LeHfgq_lAXU

image

Few amazing example images

new ad 1.png
new ad 2.png
new ad 3.png
new ad 4.png
new ad 5.png
new ad 6.png
new ad 7.png
new ad 8.png
new ad 9.png
new ad 10.png

Tutorials concepts as below

IDM-VTON: The Most Amazing Virtual Clothing Try On Application — Open Source — 1 Click Install & Use

🚨 With V5 update now works even as low as 8 GB GPUs and also free Kaggle notebook is working as well. Attention all fashion enthusiasts and tech-savvy individuals! 🚨 Get ready to have your mind blown by the most REVOLUTIONARY virtual try-on application you’ve ever seen! 🤯 IDM-VTON is here to change the game, and you won’t believe how POWERFUL and FLEXIBLE it is! 💪 Want to see your favorite anime characters or even YOURSELF in any clothing imaginable? 👀 IDM-VTON makes it possible with just a few clicks! 🖱️ You’ll be SHOCKED by the ACCURACY and QUALITY of the results! 😱 Don’t miss out on this INCREDIBLE opportunity to transform your wardrobe and unleash your creativity! 🎨 Watch this tutorial now and discover how YOU can use IDM-VTON to create MIND-BLOWING virtual fashion like never before! 🔥📹👕👖

With VTON you can try any clothing on any person with 1 click virtually. So this app will let you transfer input clothing into the input person like in science-fiction movies.

#vton #stablediffusion #virtualphotography

Scripts / Installers Download Link ⤵️

https://www.patreon.com/posts/virtual-try-on-1-103022942

How to install accurate Python, Git and FFmpeg on Windows Tutorial ⤵️

https://youtu.be/-NjNy7afOQ0

0:00 Introduction to the IDM-VTON (Virtual Try On Clothings)

1:01 How to download scripts and install on your computer

1:42 Start Windows Installation

1:52 Requirements of VTON app, e.g. Python, Git

2:27 How to verify installation has been completed accurately or not

2:57 How to download Git Large (Git LFS) to clone VTON repo accurately

3:07 How to start VTON app after installation has been completed

3:24 How to change / set your Hugging Face models cache folder

4:08 How to upload your input person image into the VTON app and how to use virtual try clothing on app

4:29 How to set VTON options

5:10 Automatic mask and crop options

6:05 How to see generation progress

6:52 The seed difference

7:02 Where the generated images saved and how the seed value is saved

7:20 How the seeding algorithm works

7:30 How to fix if Try On button is not working

8:08 How to make your own masking, mask editing example

9:24 If the mask you generated didn’t work as expected how to fix it

9:43 How to modify input image for perfect masking and output

11:33 The importance of generating multiple images

IDM-VTON: The Most Amazing Virtual Clothing Try On Application — RunPod — Massed Compute — Kaggle

In this video, I will walk through you how to install and use the amazing IDM-VTON (Virtual Try On Literally Anything) on Massed Compute, RunPod and a free Kaggle account notebook. So if you don’t have a strong GPU to use IDM-VTON locally, this is the tutorial you need. We also have a local installer that works even with 8GB GPUs. This VTON is the most amazing and advanced one. It is state of the art. Moreover, Massed Compute giving us a magnificent deal. You can also follow Massed Compute installation to install and use it on your local Linux machine.

Virtual Try-on (IDM-VTON) 1 Click Installers ⤵️

https://www.patreon.com/posts/virtual-try-on-1-103022942

Register Massed Compute From Below Link (could be necessary to use our Special Coupon for A6000 GPU for 31 cents per hour) ⤵️

https://bit.ly/Furkan-Gözükara

Full Windows Install Tutorial ⤵️

https://youtu.be/m4pcIeAVQD0

0:00 Introduction to the IDM-VTON (Virtual Try On Anything)

2:12 How to download 1 click installer scripts

2:39 How to install and use IDM Virtual Try On APP on Massed Compute cloud computing service

3:56 How to download, install and setup ThinLinc client to connect your Massed Compute virtual machine

4:15 How to setup ThinLinc client folder synchronization to transfer folders and files between your local computer and remote Massed Compute virtual machine

5:00 How to connect your remote Massed Compute virtual machine via ThinLinc

5:19 What does End Existing Session option do

5:40 How to install IDM-VTON inside Massed Compute Ubuntu desktop interface

6:01 How to transfer files from your computer to Massed Compute cloud machine

6:28 How to execute install commands on Massed Compute

7:15 Installation on Massed Compute has been completed and how to understand and verify it

7:35 How to start VTON APP after installation has been completed on Massed Compute

8:34 How to use VTON APP on my local computer via Gradio live share that runs on cloud Massed Compute machine

8:52 How to upload images onto VTON APP and use the app

9:14 How to use Open Outputs folder and where it will work

10:43 Using open outputs folder inside Massed Compute VM locally

10:54 Where the generated images are saved by default

11:05 How to fix and handle Gradio related bugs and errors that you might encounter

11:34 Importance and effect of seed

11:55 How to download all of the generated images on Massed Compute

11:42 How to install IDM-VTON (virtual try on anything) on RunPod

13:00 Special configuration I make on RunPod to get very best and powerful machines

13:48 How to add proxy port to RunPod template to connect Gradio apps through RunPod proxy

14:54 How to upload installer scripts to RunPod workspace via JupyterLab and start installing

15:30 Installation on RunPod completed so how to start the VTON app and use it

16:08 How to connect VTON app through RunPod proxy since Gradio live share didn’t work

17:04 What happens if your Gradio interface gives error while running / generating images

17:39 Where the generated images are saved on RunPod

17:50 How to download all images generated on RunPod at once

20:12 How to install and use VTON on a Free Kaggle account

22:41 How to start VTON on Kaggle with 8bit mode to make it work and start using it

26:06 How to download all of the generated images on Kaggle at once

In this comprehensive tutorial, I guide you through the setup and use of an innovative Virtual Try-On application across several platforms, including Massed Compute, RunPod, and Kaggle. Discover how to seamlessly transfer clothing and hairstyles to your images using this powerful tool. Learn how to leverage the cost-effective RTX A6000 GPU on Massed Compute at just 31 cents per hour, handle installations on RunPod despite some Gradio bugs, and utilize Kaggle’s free notebooks for your projects. This video includes step-by-step instructions, troubleshooting tips, and links to all necessary resources and scripts. Whether you’re a beginner or an experienced user, this tutorial ensures you can maximize the capabilities of the Virtual Try-On technology on any platform. Join me to enhance your digital wardrobe effortlessly!

Improving Diffusion Models for Authentic Virtual Try-on in the Wild

Paper : https://arxiv.org/pdf/2403.05139v2

Masking of dresses

Hey @yisol, this is an awesome model!

I'm trying to figure out why this is performing strangely on dresses. Specifically it seems to be just filling in the mask of the body/currently worn clothing with the dress, instead of inpainting the correct length/shape of the dress.

Here are some examples with a short floral sundress:

https://replicate.com/p/pkzppt6wphrgm0ceswqb8dj8dr
https://replicate.com/p/pgr284mnm9rgg0ceswqb8pesm4
https://replicate.com/p/yngjhan5thrgm0ceswpr1b54g0
https://replicate.com/p/twkg8appqnrgg0ceswp8a5ws6r
https://replicate.com/p/jdpt1zm11xrgm0ceswpb4fqgn0

Any recommendations to fix this?

Quote Rendering Issue

I am truly impressed with this project as it has helped me access virtual try-ons more quickly. Currently, I am encountering some issues related to images of shirts with quotes. If anyone has a solution to address this issue, I would greatly appreciate your assistance. Please help

Please refer to the attached result images for a visual reference:

  • T-shirt image: T-shirt Image
  • Result image: Result Image

Request for assistance

OSError: Can't load config for 'yisol/IDM-VTON'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'yisol/IDM-VTON' is the cor
rect path to a directory containing a config.json file

Does nothing

I installed it from pinokio, I put the images and nothing happens, nothing, I have a rtx 3060m :p, I do not get anything in the terminal to know what happens
Captura de pantalla 2024-04-26 130612
Captura de pantalla 2024-04-26 131219

Extrèmely slow + errors.

Hello,

My CMD :

Microsoft Windows [version 10.0.22631.3527]
(c) Microsoft Corporation. Tous droits réservés.

C:\Users\ejej\pinokio\api\idm-vton.git\app>conda_hook && conda deactivate && conda deactivate && conda deactivate && conda activate base && C:\Users\ejej\pinokio\api\idm-vton.git\app\env\Scripts\activate C:\Users\ejej\pinokio\api\idm-vton.git\app\env && python app.py
C:\Users\ejej\pinokio\api\idm-vton.git\app\env\lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
C:\Users\ejej\pinokio\api\idm-vton.git\app\env\lib\site-packages\transformers\utils\generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
C:\Users\ejej\pinokio\api\idm-vton.git\app\env\lib\site-packages\transformers\utils\generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
C:\Users\ejej\pinokio\api\idm-vton.git\app\env\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
The config attributes {'decay': 0.9999, 'inv_gamma': 1.0, 'min_decay': 0.0, 'optimization_step': 37000, 'power': 0.6666666666666666, 'update_after_step': 0, 'use_ema_warmup': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
C:\Users\ejej\pinokio\api\idm-vton.git\app\env\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
Some weights of the model checkpoint were not used when initializing UNet2DConditionModel:
['add_embedding.linear_1.bias, add_embedding.linear_1.weight, add_embedding.linear_2.bias, add_embedding.linear_2.weight']
Loading pipeline components...: 100%|█████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 799.24it/s]
Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().

[Start proxy] Local Sharing http://127.0.0.1:7860
Proxy Started {"target":"http://127.0.0.1:7860","proxy":"http://192.168.56.1:42421"}
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2.06it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2.61it/s]
C:\Users\ejej\pinokio\api\idm-vton.git\app\env\lib\site-packages\torch\functional.py:512: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ..\aten\src\ATen\native\TensorShape.cpp:3588.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
C:\Users\ejej\pinokio\api\idm-vton.git\app\env\lib\site-packages\diffusers\models\attention_processor.py:1259: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:455.)
hidden_states = F.scaled_dot_product_attention(
7%|███████ | 2/30 [01:40<23:14, 49.79s/it]

Thank you.

Releasing huggingface pipeline

Could the code for the Hugging Face pipeline be released? I'm asking about code to perform inference using a person's image and a clothing image.

Sampler during inference

I noticed the paper says during inference a DDPM sampler with 30 steps is used. Is this a typo and actually meant to be DDIM?

Minimum GPU memory requirements (inference)

Hi.
I was running the inference code on a 12GB 3060GPU, but I always ran into CUDA_OUT_OF_MEMORY error no matter what I do. Following is my inference command.

accelerate launch inference.py --width 384 --height 512 --num_inference_steps 30 --output_dir "result" --unpaired \
--data_dir datasets --test_batch_size 1 --guidance_scale 2.0 --mixed_precision bf16 --enable_xformers_memory_efficient_attention

I would like to know the minimum requirements for GPU configuration for running training and inference code. Furthermore, is there any other way to decrease the memory requirements besides enabling Mixed Precision, decreasing resolution and using xformers?

Gradio Demo

Hi, are you planning on releasing also the the Gradio demo script?

Anyway, thank you for the good work!

Few questions for the paper

  • Did you drop conditions during training? If yes, what rate?

  • In the appendix you list only the IP adapter and the garment unet's output as condition, did you drop the text prompt, the time_ids and all that other jazz SDXL needs when you compute CFG?

  • did you use the IPAdapter in diffusers vanilla or needed some modifications?

Thank you

How i can start?

Can I run this mode on my VPS?
I start with the full "sh nference.sh "It's over, but nothing happens anymore

Heartbeat issue

GET https://yisol-idm-vton.hf.space/heartbeat/wojrjuuvj7n 404 (Not Found)

Uncaught (in promise) {type: 'status', endpoint: '/tryon', fn_index: 2, time: Fri Apr 26 2024 15:39:50 GMT+1000 (Australian Eastern Standard Time), queue: true, …}

I am trying to implement the IDM VTON API from hugging face into a svelte page but I keep getting the above error. The heartbetat GET request isn't working and I am not sure how to work around this.

No module named 'torchvision.transforms.functional_tensor

Hello i get this error, can any 1 have idea? thanks.
`
C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app>conda_hook && conda deactivate && conda deactivate && conda deactivate && conda activate base && C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\Scripts\activate C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env && python app.py
C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_p
Microsoft Windows [Version 10.0.22631.3447]
(c) Microsoft Corporation. Tüm hakları saklıdır.

C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app>conda_hook && conda deactivate && conda deactivate && conda deactivate && conda activate base && C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\Scripts\activate C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env && python app.py
C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\transformers\utils\generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\transformers\utils\generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
_torch_pytree._register_pytree_node(
C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
The config attributes {'decay': 0.9999, 'inv_gamma': 1.0, 'min_decay': 0.0, 'optimization_step': 37000, 'power': 0.6666666666666666, 'update_after_step': 0, 'use_ema_warmup': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils.pytree.register_pytree_node instead.
torch.utils.pytree.register_pytree_node(
Some weights of the model checkpoint were not used when initializing UNet2DConditionModel:
['add_embedding.linear_1.bias, add_embedding.linear_1.weight, add_embedding.linear_2.bias, add_embedding.linear_2.weight']
Traceback (most recent call last):
File "C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\app.py", line 102, in
openpose_model = OpenPose(0)
File "C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\preprocess\openpose\run_openpose.py", line 33, in init
self.preprocessor = OpenposeDetector()
File "C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\preprocess\openpose\annotator\openpose_init
.py", line 52, in init
from basicsr.utils.download_util import load_file_from_url
File "C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\basicsr_init
.py", line 4, in
from .data import *
File "C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\basicsr\data_init.py", line 22, in
dataset_modules = [importlib.import_module(f'basicsr.data.{file_name}') for file_name in dataset_filenames]
File "C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\basicsr\data_init
.py", line 22, in
dataset_modules = [importlib.import_module(f'basicsr.data.{file_name}') for file_name in dataset_filenames]
File "C:\YapayZeka\pinokio\bin\miniconda\lib\importlib_init
.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\basicsr\data\realesrgan_dataset.py", line 11, in
from basicsr.data.degradations import circular_lowpass_kernel, random_mixed_kernels
File "C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app\env\lib\site-packages\basicsr\data\degradations.py", line 8, in
from torchvision.transforms.functional_tensor import rgb_to_grayscale
ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'

(env) (base) C:\YapayZeka\pinokio\api\ust-degis-idm-vton.git\app>
[Start proxy] Local Sharing {{input.event[0]}}
Proxy Started {"target":"{{input.event[0]}}","proxy":"http://192.168.56.1:42421"}
`

About agnostic-mask

Great job. I tried to execute the inference code, but found that there was no agnostic-mask data. Do you have plans to release this data?

Lower Body Clothes?

Is it working with lower body clothing items or dresses? I didnt see it in hugging face space

What is the minimum memory requirment?

I have 4060 8GB VRAM + 8GB shared RAM and it fails all the time with CUDA memory error.

Is there any setting which I can modify so it can run on lower requirements. I already tried 360 x 480 image size and same memory issue.

Is weights link broken?

I tried to download the weights (openpose, densepose, etc) but the link seems to be broken. (Link can be seen in the image below that show section of README)

image

Will there be ComfyUI implementation?

I love what you have made here. I designed a workflow that gets close to this but not like the results you have. Are you planning on integrating this with ComfyUI as custom nodes??

My Youtube is AIFUZZ and we would gladly work with you to help test any new iterations

Need Some Modification

Hi we need some modification Please contact me on +918320362419 whatsapp and i will pay for changes Please contact asap

Training Code

Good Morning. Your model seems great, when will you release the training code for it?

Train and inference code

Hi, very impressive work, your demo looks amazing. Do you have any plan for releasing train and inference code along with the base architecture?

Could I use my own pictures?

Nice job! I have used inference.py to test VITON-HD dataset. However, I wonder if the codes support to use my own pictures to generate images ( Instead of by using demo 🙂)?

Inference for Dresscode

Will you release the inference code for the Dresscode dataset? If yes, when would that be?

Training questions

Hey, great work! Quick question on training.

I was wondering how you're fitting two SDXL UNets (garment UNet and tryon UNet) on a single A800 with batch size 24/4=6 (assuming 4xA800 in total). I see you're using FP16 models, but are you doing any optimizations to bring memory down, like precomputing embeddings / features, 8bit adam or gradient accumulation? I'm trying to reproduce training, but can only fit 3 samples at 1024x768 resolution on 80GB VRAM during training and a single step takes ~1.3 seconds on a H100. I'm already doing the above tricks (8bit adam, precomputing VAE embeddings, frozen garment unet).

Also curious about training speed if you can share. Thanks!

Does this work with Lower-Body and Dresses

Amazing Work releasing this, I just had a question does this work with lower-body and dresses as garment input? I was not able to find any qualitative examples on the paper.

Load model from C:\tut\IDM-VTON\ckpt/humanparsing/parsing_atr.onnx failed:Protobuf parsing failed

Any suggestions please.

I noticed it is only using RAM and not VRAM, not sure why?

(venv) C:\tut\IDM-VTON>python gradio_demo/app.py
The config attributes {'decay': 0.9999, 'inv_gamma': 1.0, 'min_decay': 0.0, 'optimization_step': 37000, 'power': 0.6666666666666666, 'update_after_step': 0, 'use_ema_warmup': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Some weights of the model checkpoint were not used when initializing UNet2DConditionModel:
 ['add_embedding.linear_1.bias, add_embedding.linear_1.weight, add_embedding.linear_2.bias, add_embedding.linear_2.weight']
Traceback (most recent call last):
  File "C:\tut\IDM-VTON\gradio_demo\app.py", line 95, in <module>
    parsing_model = Parsing(0)
  File "C:\tut\IDM-VTON\.\preprocess\humanparsing\run_parsing.py", line 20, in __init__
    self.session = ort.InferenceSession(os.path.join(Path(__file__).absolute().parents[2].absolute(), 'ckpt/humanparsing/parsing_atr.onnx'),
  File "C:\tut\IDM-VTON\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "C:\tut\IDM-VTON\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 452, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidProtobuf: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from C:\tut\IDM-VTON\ckpt/humanparsing/parsing_atr.onnx failed:Protobuf parsing failed.

(venv) C:\tut\IDM-VTON>pip list

Package                   Version
------------------------- ------------------
absl-py                   2.1.0
accelerate                0.25.0
addict                    2.4.0
aiofiles                  23.2.1
altair                    5.3.0
annotated-types           0.6.0
antlr4-python3-runtime    4.9.3
anyio                     4.3.0
attrs                     23.2.0
av                        12.0.0
basicsr                   1.4.2
bitsandbytes              0.39.0
certifi                   2024.2.2
charset-normalizer        3.3.2
click                     8.1.7
cloudpickle               3.0.0
colorama                  0.4.6
coloredlogs               15.0.1
contourpy                 1.2.1
cycler                    0.12.1
diffusers                 0.25.0
einops                    0.7.0
exceptiongroup            1.2.1
fastapi                   0.110.3
ffmpy                     0.3.2
filelock                  3.14.0
flatbuffers               24.3.25
fonttools                 4.51.0
fsspec                    2024.3.1
future                    1.0.0
fvcore                    0.1.5.post20221221
gradio                    4.24.0
gradio_client             0.14.0
grpcio                    1.63.0
h11                       0.14.0
httpcore                  1.0.5
httpx                     0.27.0
huggingface-hub           0.22.2
humanfriendly             10.0
idna                      3.7
imageio                   2.34.1
importlib_metadata        7.1.0
importlib_resources       6.4.0
iopath                    0.1.10
Jinja2                    3.1.3
jsonschema                4.22.0
jsonschema-specifications 2023.12.1
kiwisolver                1.4.5
lazy_loader               0.4
lightning-utilities       0.11.2
lmdb                      1.4.1
Markdown                  3.6
markdown-it-py            3.0.0
MarkupSafe                2.1.5
matplotlib                3.8.4
mdurl                     0.1.2
mpmath                    1.3.0
networkx                  3.3
numpy                     1.26.4
omegaconf                 2.3.0
onnxruntime               1.16.2
opencv-python             4.9.0.80
orjson                    3.10.2
packaging                 24.0
pandas                    2.2.2
pillow                    10.3.0
pip                       24.0
platformdirs              4.2.1
portalocker               2.8.2
protobuf                  4.25.3
psutil                    5.9.8
pycocotools               2.0.7
pydantic                  2.7.1
pydantic_core             2.18.2
pydub                     0.25.1
Pygments                  2.17.2
pyparsing                 3.1.2
pyreadline3               3.4.1
python-dateutil           2.9.0.post0
python-multipart          0.0.9
pytorch-triton            0.0.1
pytz                      2024.1
pywin32                   306
PyYAML                    6.0.1
referencing               0.35.0
regex                     2024.4.28
requests                  2.31.0
rich                      13.7.1
rpds-py                   0.18.0
ruff                      0.4.2
safetensors               0.4.3
scikit-image              0.23.2
scipy                     1.11.1
semantic-version          2.10.0
setuptools                63.2.0
shellingham               1.5.4
six                       1.16.0
sniffio                   1.3.1
starlette                 0.37.2
sympy                     1.12
tabulate                  0.9.0
tb-nightly                2.17.0a20240501
tensorboard-data-server   0.7.2
termcolor                 2.4.0
tifffile                  2024.4.24
tokenizers                0.15.2
tomli                     2.0.1
tomlkit                   0.12.0
toolz                     0.12.1
torch                     2.0.1+cu118
torchaudio                2.0.2+cu118
torchmetrics              1.2.1
torchvision               0.15.2+cu118
tqdm                      4.66.1
transformers              4.36.2
typer                     0.12.3
typing_extensions         4.11.0
tzdata                    2024.1
urllib3                   2.2.1
uvicorn                   0.29.0
websockets                11.0.3
Werkzeug                  3.0.2
yacs                      0.1.8
yapf                      0.40.2
zipp                      3.18.1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.