CAM loss is degrading the results about ugatit HOT 6 CLOSED

I have had the same experiences.
In my case, there are basically two reasons for this:

1. The task of your project doesn't require much shape changes, or even hates to do so.
2. Contents of the dataset you are using are not camera images that are RGB based, which might make the CAM part acting unexpectedly.

As for your guess, I don't see the reason for CAM loss to fail in light mode as only an extra pooling is performed for light mode. Also I tested a similar task as proposed in the original paper with different datasets, they worked in light mode (but not as good as heavy mode for sure).

Hope my answer helps, cheers.

from ugatit.

Thank you for your response! I really appreciate it.

Yeah, your points seem valid in my case too. I also think images I'm using don't require much shape changes. And, yes, I'm working with single channel images. So, CAM seems to be an overkill.

Moreover, any thoughts on the effect of CAM using the paired dataset instead of unpaired dataset? Right now, I think CAM will be more effective with unpaired dataset instead of paired one.

Glad that my answer helps. CAM, in my opinion, will certainly help for paired datasets. By recalling how CAM works, the reason can be obvious: the network knows which activation map to focus using CAM. Problem is that you might need to make sure CAM highlights the exact spatial position as you want to.

from ugatit.

2. RGB based

Hi, why is the CAM can only work with RGB based images? Do you have any insights?

from ugatit.

Thank you for your response! I really appreciate it.

Yeah, your points seem valid in my case too. I also think images I'm using don't require much shape changes. And, yes, I'm working with single channel images. So, CAM seems to be an overkill.

Moreover, any thoughts on the effect of CAM using the paired dataset instead of unpaired dataset? Right now, I think CAM will be more effective with unpaired dataset instead of paired one.

from ugatit.

2. RGB based

Hi, why is the CAM can only work with RGB based images? Do you have any insights?

Simply declaring CAM only works with RGB might be wrong, at least not accurate.
By 'CAM might act unexpectedly', I merely want to say that CAM embedded in a cycle-GAN style network may not act as you would hope to, since CAM part is upgraded with the guidance from Discriminator in an unsupervised manner.
BTW, I believe that this issue can now be closed, right? @sudarshanregmi

from ugatit.

Oh, yes @bcccat. Thanks for your help! 👍

from ugatit.