tryonlabs / tryondiffusion Goto Github PK

The paper does not mention clearly how noise augmentation levels are applied. Different noise augmentation levels for different items in the batch can also be experimented with. As of now, I have kept the noise augmentation level the same across the batch.

I am looking into training the model. But I cannot understand the format of the dataset.

Can you tell me what the different abbreveated folders stand for? The folders: ip, jp, ia, ic and jg. Can probably get image-parse from ip but the rest are a mistery.

Pooled pose embeddings must be summed with positional encodings of diffusion time step t, and noise augmentation levels tna. As explained at the end of section 3.2 in the paper.

1.dataloader_train.py line 72-75，missed T.ToTensor(), which is include /255. T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)), only ensure -0.5 and /0.5. recommend use PIL to preprocess image instead of use opencv.
2. diffusion.py line 79-83, def add_noise_to_img(self, img, t): return (sqrt_alpha_timestep * epsilon) + (sqrt_one_minus_alpha_timestep * epsilon), epsilon, which is should be return (sqrt_alpha_timestep * img) + (sqrt_one_minus_alpha_timestep * epsilon), epsilon.

Pass device to both the UNets while initializing them, and use self.device to pass it to the GaussianSmoothening function in AttentionPool1d.

The function draw_bodypose in utils.py is not included in body_pose.py

It is required to draw the estimated pose.

Recommended Change: Include draw_bodypose in body_pose.py and call it after calling body_estimation.

I am creating this issue to remember to experiment with leaky relu instead of relu while creating pose embeddings.

please add a link to the discord server

I made a little mistake while implementing the UNet architecture. Instead of concatenating, I added(elementwise summation) the channels in UNet's skip connections.

Ideally, we must concatenate them channel-wise and reduce the channels during convolution. It's a straightforward fix. I am creating the issue to keep track of and fix the problem before training.

Thanks for the excellent work!
I want to know that why dim_head is set to 1 in self-attention and cross-attention?

Is this project available for use or is it under construction? Plus if it is available kindly guide how to run this since there is only setup file to download dependencies only.

Schedule learning rate as given in paper.

Hello, the noise here seems to be wrong, the first item should be img instead of epsilonhttps://github.com/kailashahirwar/tryondiffusion/blob/main/diffusion.py#L84

I want to know how to use this in production.
Is this just a model implementation? If it is the implementation of the architecture then how can we train it over our dataset?

can you help me out in finding the garment landmarks keypoints? and what are the techniques that u have used to get the garment landmark keypoints

The paper says, "We follow classifier-free guidance and train our models with conditioning dropout: conditional inputs are set to 0 for 10% of training time."

This means during 10% of the training time, only Person UNet is to be trained, without any cross-attention, or self attention or anything in conditional inputs.

Maybe I am not familiar with the concept but how would it work without RGB-agnostic images, or how 6 channels would be passed? Do we make values 0 for RGB agnostic images? Any comments are welcome.

There is a small bug in train.py script of human and garment pose models.

The bug is present in below lines:

for test_keypoints in test_dataloader:
            model.eval()
            test_predictions, _ = model(test_keypoints)

The correction should be:

for test_keypoints, _ in test_dataloader:
            model.eval()
            test_predictions, _ = model(test_keypoints)