Problems about groups about stand-alone-self-attention HOT 4 OPEN

The 'groups' are not like the groups for group convolutions. In this case, each of the multi-attention heads takes in the full input dimension (not 1/n_groups of it). Computationally, this is done as one convolution and then reshaped to separate the heads.

from stand-alone-self-attention.

However, it seems like not to separate k_out, v_out and q_out into groups gives the same out results in the above codes. So what does the group parameter exactly do?

from stand-alone-self-attention.

Seems like q_out and k_out should be 'matrix-multiplied' at dim=2 (the one with the size self.out_channels // self.groups).

from stand-alone-self-attention.

According to the listed code lines, it seems that self.groups has no effect on the result. I beleive no matter how to set self.groups, the code can only realize the case "group=1".

from stand-alone-self-attention.