blackfeather-wang / infopro-pytorch Goto Github PK

Hi. Thanks for the great idea and sharing the code.
I have a question. You changed the position of runner.optimizer.zero_grad() from after_train_iter to before_train_iter, refererence to

I was wondering what this change will do and how does it affect the training and model performance.
I adopted your idea to UNet in medical segmentation but missed the aforementioned change. Yet the training converges correctly as expected. Now I realized the change and like to know its supposed effect.
Thank you again.

Hi. This work is really a fantastic job! I want to do some further research based on it in order to reduce the required resources. I noticed that you've provided memory usage in your paper. May I ask how did you measure the GPU footprint?
Thanks very much. Looking forward to your reply.

Hi, @blackfeather-wang, I would like to ask if there is any discrepancy between your code implementation and the description of the paper. In your paper, you say that we are going to minimize $\mathcal{L}{contrast}$, and the form of $\mathcal{L}{contrast}$ is as follows.

However, after I read your code, I find that your code seems to minimize $-\mathcal{L}_{contrast}$, and the consequence is that you're not maximizing the lower bound of $I(h,y)$, instead you're minimizing the lower bound of $I(h,y)$, can you explain that?

Thanks for your awesome work! I'm very interested in mutual information estimation used in your paper.
According to Appendix-G and your finding (figure-6), you train an auxiliary classifier to estimate I(h, y) and end2end supervised training retains all task-relevant information.
From my perspective, it shows that we don't need to build a classifier upon the final feature map and deploying the classifier (trained on feature maps from many layers) to the first feature map is enough.
I'm not sure I understand this correctly. Would you help me clarify this?