tryonlabs / tryondiffusion Goto Github PK
View Code? Open in Web Editor NEWTryOnDiffusion: A Tale of Two UNets Implementation
TryOnDiffusion: A Tale of Two UNets Implementation
The paper does not mention clearly how noise augmentation levels are applied. Different noise augmentation levels for different items in the batch can also be experimented with. As of now, I have kept the noise augmentation level the same across the batch.
I am looking into training the model. But I cannot understand the format of the dataset.
tryondiffusion/tryondiffusion/trainer.py
Lines 10 to 20 in d471b91
Can you tell me what the different abbreveated folders stand for? The folders: ip
, jp
, ia
, ic
and jg
. Can probably get image-parse
from ip
but the rest are a mistery.
Pooled pose embeddings must be summed with positional encodings of diffusion time step t, and noise augmentation levels tna. As explained at the end of section 3.2 in the paper.
1.dataloader_train.py line 72-75,missed T.ToTensor(), which is include /255. T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)), only ensure -0.5 and /0.5. recommend use PIL to preprocess image instead of use opencv.
2. diffusion.py line 79-83, def add_noise_to_img(self, img, t): return (sqrt_alpha_timestep * epsilon) + (sqrt_one_minus_alpha_timestep * epsilon), epsilon, which is should be return (sqrt_alpha_timestep * img) + (sqrt_one_minus_alpha_timestep * epsilon), epsilon.
Pass device
to both the UNets while initializing them, and use self.device
to pass it to the GaussianSmoothening
function in AttentionPool1d.
The function draw_bodypose in utils.py is not included in body_pose.py
It is required to draw the estimated pose.
Recommended Change: Include draw_bodypose in body_pose.py and call it after calling body_estimation.
I am creating this issue to remember to experiment with leaky relu instead of relu while creating pose embeddings.
please add a link to the discord server
I made a little mistake while implementing the UNet architecture. Instead of concatenating, I added(elementwise summation) the channels in UNet's skip connections.
Ideally, we must concatenate them channel-wise and reduce the channels during convolution. It's a straightforward fix. I am creating the issue to keep track of and fix the problem before training.
Thanks for the excellent work!
I want to know that why dim_head is set to 1 in self-attention and cross-attention?
Is this project available for use or is it under construction? Plus if it is available kindly guide how to run this since there is only setup file to download dependencies only.
Schedule learning rate as given in paper.
Hello, the noise here seems to be wrong, the first item should be img instead of epsilonhttps://github.com/kailashahirwar/tryondiffusion/blob/main/diffusion.py#L84
I want to know how to use this in production.
Is this just a model implementation? If it is the implementation of the architecture then how can we train it over our dataset?
can you help me out in finding the garment landmarks keypoints? and what are the techniques that u have used to get the garment landmark keypoints
The paper says, "We follow classifier-free guidance and train our models with conditioning dropout: conditional inputs are set to 0 for 10% of training time."
This means during 10% of the training time, only Person UNet is to be trained, without any cross-attention, or self attention or anything in conditional inputs.
Maybe I am not familiar with the concept but how would it work without RGB-agnostic images, or how 6 channels would be passed? Do we make values 0 for RGB agnostic images? Any comments are welcome.
There is a small bug in train.py script of human and garment pose models.
The bug is present in below lines:
for test_keypoints in test_dataloader:
model.eval()
test_predictions, _ = model(test_keypoints)
The correction should be:
for test_keypoints, _ in test_dataloader:
model.eval()
test_predictions, _ = model(test_keypoints)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.