vasishta / linear-attention-recurrent-neural-network Goto Github PK
View Code? Open in Web Editor NEWThis project forked from guillaume-chevalier/linear-attention-recurrent-neural-network
A recurrent attention module consisting of an LSTM cell which can query its own past cell states by the means of windowed multi-head attention. The formulas are derived from the BN-LSTM and the Transformer Network. The LARNN cell with attention can be easily used inside a loop on the cell state, just like any other RNN.
Home Page: http://www.neuraxio.com/en/
License: MIT License