Wrong input for generating self.D in models/cgan_model.py line 44 about deblurgan-tf HOT 7 CLOSED

Thank you @Agchai52 for you careful code review review.
https://github.com/KupynOrest/DeblurGAN/blob/39e8a4b408b90d0ef98608c5c4562eae3e184251/models/losses.py#L157-L167

I checked the codes which you refer.
In the case of discriminator, the input is [generated image, real image].

In Conditional GAN, the blurry image is the auxiliary information for (not both) generator only.
After condition of blur image, generator creates fake image.
Then, discriminator try to classify the fake image and the real (sharp) image.

However, in the review process, i found another bug for generator loss function.
18a7f36

Thank you.

from deblurgan-tf.

Thank you for your response!

Both Discriminator and Generator do condition on auxiliary information (blurry).

where x is auxiliary imformation or the observed image (blurry image), y is the sharp image (real image) and z is the random noise. Both in the first term and second term, the inputs of D contain x (blurry image).

Eq. (1) comes from paper "Image-to-Image Translation with Conditional Adversarial Networks" by Isola et al. This paper is one of the most important references of DeblurGAN. We can also check Eq. (2) in the original conditional GAN paper "Conditional Generative Adversarial Nets" by Mirza et al. Although notation is different, the same conclusion can be drawn.

from deblurgan-tf.

I understand the equation and the condition of blur image is important also in discriminator.
However, first, the discriminator loss need two result image: real_img and generated_img.

How can we calculate D(x,y) ??
Does it means ??

G_input = tf.concat([self.G, self.input['blur_img'], axis=0)
D_input = tf.concat([self.input['real_img'], self.input['blur_img']], axis=0)
self.D = discriminator(tf.concat([G_input, D_input] , axis=0)

Meanwhile, the reason why i concat generated_image and real image is to output two probability (first_term and second term in (1))

from deblurgan-tf.

However I found that the your reference DeblurGAN code (pytorch, original author) do not concat the blur image into the input of discriminator

from deblurgan-tf.

That a very quick response!

I just checked the link your shared. The problem is actually on DeblurGAN. You just follow their code. I didn't dive into their function. Sorry about that. (the pytorch DeblurGAN is too painful to read, forgive me...)

The way to use auxiliary info is exactly what you show. Just concatinate the blurry after the real and the generated. To get D(x,y) and D(x, G(x,z)), we need to feed them into Discrimiantor separately, rather than together.

You can check this link for more details: https://github.com/yenchenlin/pix2pix-tensorflow/blob/master/model.py
Line 75-83.

This the tf version of Image-to-image.

from deblurgan-tf.

Thank you. I didn't check the original pix2pix codes carefully, but your comments are really helpful to me.

Your comment is right, most of GAN models are implement to feed generator and discriminator separately.
However, after i read the DeblurGAN paper, the model doesn't need to do.
So i concat the self.G and self.input['real_img'] and feed together.
(The model use batch_size=1. Memory is enough but separate feed makes the training slow.
Of course, to use gpu fully, i will additionally implement the data_feeder module with tf.data)

from deblurgan-tf.

So, DeblurGAN doesn't use conditional GAN as the claimed.
That makes sense!
Thank you for your reply!
Great code!

from deblurgan-tf.