Problem about GMPool_I about gmt HOT 2 CLOSED

Hi,
thank you for your interest and great question!

This is not a bug, but rather our purpose.
In specific, we indeed use the col-wise softmax to make the overall architecture as powerful as the WL test. Considering the example of Graph Isomorphism Network (GIN) that uses the summation over all node representations for approximating the WL test. You can check the details in Section 3.3, Section 3.4, and Proof in Appendix A of our paper.

You can also use the row-wise softmax without using the cluster option in Line #29 of layers.py, but we found that this row-wise GMT mostly underperforms our proposed col-wise GMT.

Finally, the above result only happens in the final layer, where we reduce all remaining nodes into one particular node with one seed vector, thus this is the same as the sum pooling. You are correct. However, please note that, when we reduce n nodes into k different nodes, the col-wise softmax correctly assigns k cluster values (summed to one) for each node; the matrix is not the all-1 matrix.

from gmt.

Hi, thank you for your interest and great question!

This is not a bug, but rather our purpose. In specific, we indeed use the col-wise softmax to make the overall architecture as powerful as the WL test. Considering the example of Graph Isomorphism Network (GIN) that uses the summation over all node representations for approximating the WL test. You can check the details in Section 3.3, Section 3.4, and Proof in Appendix A of our paper.

You can also use the row-wise softmax without using the cluster option in Line #29 of layers.py, but we found that this row-wise GMT mostly underperforms our proposed col-wise GMT.

Finally, the above result only happens in the final layer, where we reduce all remaining nodes into one particular node with one seed vector, thus this is the same as the sum pooling. You are correct. However, please note that, when we reduce n nodes into k different nodes, the col-wise softmax correctly assigns k cluster values (summed to one) for each node; the matrix is not the all-1 matrix.

Get it. Thanks for your reply. While reducing n nodes into k different nodes, col-wise softmax is necessary (same as DiffPool). But, to my knowledge and experiments, in hiv datasets, a mean or sum global pooling doesn't perform better than GlobalAttentionPool which assign nodes different weights. Maybe the method to get node weights of the row-wise GMT doesn't good enough. I think GMPool_l+SelfAtt+ some better global pooling may be work. I'all try it.

Thanks again.

from gmt.