sled-group / cyclenet Goto Github PK

great job! I have a question that is ok that make images as condition instead of text? for example, as for original dataset the dataset structure is like this:

fill50k/train.jsonl
fill50k/images/X.png
fill50k/conditioning_images/X.png

can i use the conditioning_images as condition, and change the relevent part of the code in here:

class MyDataset(Dataset):
    def __init__(self):
        self.data = []
        with open('./training/cfill50k/prompt.json', 'rt') as f:
            for line in f:
                self.data.append(json.loads(line))

    def __len__(self):
        return len(self.data)

    def __getitem__(self, idx):
        item = self.data[idx]

        image_filename = item['image']
        source = item['source']
        target = item['target']

        image = cv2.imread('./training/cfill50k/' + image_filename)

        # Do not forget that OpenCV read images in BGR order.
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

        image = (image.astype(np.float32) / 127.5) - 1.0

        return dict(jpg=image, source=source, txt=target)

Thank you!

Your work looks impressive. I would like to ask, in the summer2winter task, what kind of source prompts and target prompts are provided?

Thanks for great work!

I followed the example provided with cfill dataset.
-> https://github.com/sled-group/CycleNet/blob/main/docs/train.md

I found 2 differences with the original works.

1. Training with batch size = 1 cost over 30GB VRAM, which is different in paper (batch size = 4 cost 27.9GB VRAM).

2. My ckpt file (saved in training) is 13.7 GB, which is different in provided ckpt (summer2winter.ckpt = 6.7 GB)

Would you explain how to reproduce the original works?

hello, great work!
How to train the model based on Scene/Object-Level Manipulation dataset or ManiCups dataset?
@Mars-tin @Roihn @h6kplus @SihanXU
Or how to sample based on pre-trained models?

In the 'docs/train.md' section, a link to the CycleFill50k dataset is provided, but it leads to a 404 error page. Could you please provide the dataset or the 'prompt.json' file? I have located the checkpoint, but I am looking to train with my own data.

I am using 4 GPUS A6000 to train the model based on cyclefill50, and the epoch is now 6. However, the result is not good:

@Mars-tin @SihanXU

Hi, great job.

as mentioned, will you release the full training code of winter to summer?

I read your paper, it's an amazing work and I impressed with the results. And I would like to try this model for generation on my samples, but I don't find the weights for model. Can you provide pre-trained weights?

great work, looking forward to it, but the paper link not work , when will the paper release?

I tried the summer2winter model，but the results are out of expectations，the parameters as follows:

process(input_image, 'winter', 'summer', num_samples=1, image_resolution=512, ddim_st eps=100, guess_mode=False, strength=1.0, scale=9.0, seed=1000, eta=1.0, denoise_strength=0.5)

input image:

output image