Question about discriminator about gigagan-pytorch HOT 8 CLOSED

I think the multi scale input may not be produced by resize, it's produced from every to_rbg layer like https://github.com/JiauZhang/GigaGAN/blob/main/model.py#L254

from gigagan-pytorch.

@gooobot ohh got it! so the generator will need to output all the rgb at all stages, and be able to pass it into the discriminator. i can get that done tomorrow morning, thank you! 🙏
if a real image taken from the dataset is fed into the discriminator, then it would be resized as the code does now?

Yes, the generator will output images with [4x, 8x, ..., 32x, 64x] resolution when use_multi_scale is enabled.

That's what I mean! @lucidrains Thank you for you great effort!

from gigagan-pytorch.

@potato123-hash hello! so i'm actually doing that automatically here. let me know if i misunderstood that part of the paper

from gigagan-pytorch.

@gooobot ohh got it! so the generator will need to output all the rgb at all stages, and be able to pass it into the discriminator. i can get that done tomorrow morning, thank you! 🙏

if a real image taken from the dataset is fed into the discriminator, then it would be resized as the code does now?

from gigagan-pytorch.

@gooobot ohh got it! so the generator will need to output all the rgb at all stages, and be able to pass it into the discriminator. i can get that done tomorrow morning, thank you! 🙏

if a real image taken from the dataset is fed into the discriminator, then it would be resized as the code does now?

Yes, the generator will output images with [4x, 8x, ..., 32x, 64x] resolution when use_multi_scale is enabled.

from gigagan-pytorch.

@potato123-hash @gooobot ok, makes sense! will get this all fixed tomorrow! thank you for raising this issue

from gigagan-pytorch.

@potato123-hash do you want to try 0.0.16 and see if that fits your intuition?

import torch
from gigagan_pytorch.gigagan_pytorch import (
    TextEncoder,
    Generator,
    Discriminator,
    StyleNetwork
)

text_encoder = TextEncoder(
    dim = 512,
    depth = 2
).cuda()

discr = Discriminator(
    dim = 64,
    dim_max = 512,
    image_size = 256,
    text_encoder = text_encoder,
    use_glu = True,
    num_skip_layers_excite = 4,
    unconditional = False
).cuda()

style_network = StyleNetwork(
    dim = 64,
    depth = 4,
    dim_text_latent = text_encoder.dim
).cuda()

generator = Generator(
    dim = 64,
    style_network = style_network,
    text_encoder = text_encoder,
    image_size = 256,
    dim_max = 512,
    use_glu = True,
    num_skip_layers_excite = 4
).cuda()

# mock data

real_images = torch.randn(1, 3, 256, 256).cuda()
texts = ['a happy dog wagging her tail']

# generator

image, rgbs = generator(
    texts = texts,
    batch_size = 1,
    return_all_rgbs = True
)

# discriminator

logits, *_ = discr(
    image,
    rgbs,
    real_images = real_images,
    texts = texts
)

from gigagan-pytorch.

feel free to reopen if it isn't resolved!

from gigagan-pytorch.