Question about Figure 2(a) about how-do-vits-work HOT 6 CLOSED

Hi @iumyx2612,

I believe a more appropriate interpretation of this figure is that the convolutional layers consistently amplify high-frequency components. Consequently, a significant amount of high-frequency information remains in the representations of deeper layers. My emphasis was on this trend of changes.

from how-do-vits-work.

Hi @iumyx2612,

I believe a more appropriate interpretation of this figure is that the convolutional layers consistently amplify high-frequency components. Consequently, a significant amount of high-frequency information remains in the representations of deeper layers. My emphasis was on this trend of changes.

So looking at the figure, we can't say that early layers in a ConvNet contains many low frequency components right?

from how-do-vits-work.

It is not easy to simply compare the amount of low-frequency components from different depths of layers. Instead, I'd like to say that having many low-frequency components in a representation does not necessarily mean the layers are learning low-frequency information in this case.

from how-do-vits-work.

It is not easy to simply compare the amount of low-frequency components from different depths of layers. Instead, I'd like to say that having many low-frequency components in a representation does not necessarily mean the layers are learning low-frequency information in this case.

Oooh, thank you I got it!

from how-do-vits-work.

I was reading the paper: MogaNet and having difficult understanding some part in the paper. Some of the explanations contradict to my understanding so I would like to exchange something with you (through emails or other platform since it's not relevant to this github) since some parts of the paper take inspiration from your work. Is it okay? Sorry if this bothers you

from how-do-vits-work.

I am happy to do it. Please feel free to send me an email at [email protected] or [email protected].

from how-do-vits-work.