i find the decoder which in your implementation is conv2d,and it is different with MPLDecoder which used in segformer paper ? about segformer-pytorch HOT 7 OPEN

the MixFeedForward is alse only used conv2d @ lucidrains

from segformer-pytorch.

a conv2d of kernel size of 1 is equivalent to a linear layer. I did it to avoid transposes

from segformer-pytorch.

a conv2d of kernel size of 1 is pointwise conv,and it's parameters reused for all point, did i make a mistake?

from segformer-pytorch.

from segformer-pytorch.

a pointwise conv is a linear layer applied on the channel axis (dimension 1)

from segformer-pytorch.

maybe use the pointwise conv to change feature channels is faster,but i think the grouped conv with groups=channels and kernelsize = HxW equal to linear

from segformer-pytorch.

I don't quite follow, I don't think they are projecting along the sequence dimension (height x width)

from segformer-pytorch.