Recently, I have been working on Seq2Seq Learning and I decided to prepare a series of tutorials about Seq2Seq Learning from a simple Multi-Layer Perceptron Neural Network model to Encoder Decoder Model with Attention.

• Our aim is to code an Encoder Decoder Model with Attention.
• However, I would like to develop the solution by showing the shortcomings of other possible approaches.
• Therefore, in the first 2 parts, we will observe that initial models have their own weakness.
• We also understand why Encoder Decoder paradigm is so successful.

Part A: AN INTRODUCTION TO SEQ2SEQ LEARNING AND A SAMPLE SOLUTION WITH MLP NETWORK

Part B: SEQ2SEQ LEARNING WITH RECURRENT NEURAL NETWORKS (LSTM)

Part C: SEQ2SEQ LEARNING WITH A BASIC ENCODER DECODER MODEL

Part D: SEQ2SEQ LEARNING WITH AN ENCODER DECODER MODEL + TEACHER FORCING

Part E: SEQ2SEQ LEARNING WITH AN ENCODER DECODER MODEL FOR VARIABLE INPUT AND OUTPUT SIZE

Part F: SEQ2SEQ LEARNING WITH AN ENCODER DECODER MODEL + TEACHER FORCING FOR VARIABLE INPUT AND OUTPUT SIZE

Part G: SEQ2SEQ LEARNING WITH AN ENCODER DECODER MODEL + BAHDANAU ATTENTION

Part H: SEQ2SEQ LEARNING WITH AN ENCODER DECODER MODEL + LUONG ATTENTION

• tf.keras.layers.LSTM official website

• A ten-minute introduction to sequence-to-sequence learning in Keras by Francois Chollet

• How to Develop an Encoder-Decoder Model with Attention in Keras by Jason Brownlee

• https://medium.com/deep-learning-with-keras/part-a-introduction-to-seq2seq-learning-a-sample-solution-with-mlp-network-95dc0bcb9c83