Based on the paper "On the Relationship between Self-Attention and Convolutional Layers" by Jean-Baptiste Cordonnier, Andreas Loukas, and Martin Jaggi, and its reproduction by wandb.ai.
Work in-progress!
- Finish going through the paper and devise a deconvolution experiment
- Build SA layer for 1D