mask convolution about convmae HOT 3 CLOSED

Thanks for your interest.
There are two ways to verify information leakage or not.

1. Please check the reconstruction loss. It can empirically prove there is no information leakage happen inside ConvMAE.
   https://drive.google.com/file/d/1Je9ClIGCQP43xC3YURVFPnaMRC0-ax1h/view?usp=sharing
2. Inside stage 1 and stage 2, spatial information aggregation happen at depthwise convolution operation. Masking on DW can prevent information leakage. Adding mask operations on other operator is redundant.

from convmae.

Hi @gaopengpjlab and @Alpha-VL ,

I just find out this thread discussing the issue I was curious to know. I can understand that the local attention only works on dw-conv but there is still an FFN after it. It is possible that FFN mixes some information from the skip connection branch and leads to the leakage. For the extreme case, the possibility of network learning to pass the original image throughout the all model by skip connection is still held. Let me know if I misunderstand anything. Thanks!

from convmae.

Yeach, I have the same doubt.

from convmae.