An implementation of Transfomer.

This code are depend on the following.

• python==3.6.5
• pytorch==1.1.0
• torchtext==0.3.1

git clone https://github.com/t080/pytorch-transformer.git
cd ./pytorch-transformer
pip install -r requirements.txt

The --train and --valid options receive the path to a data file for training and validation, respectively. The data file must be tab-separated values (TSV) format. If you need to use GPU, please set the --gpu option.

python train.py \
  --train ./data/sample_train.tsv \
  --valid ./data/sample_valid.tsv \
  --savedir ./checkpoints \
  --gpu

The --model option receives the path to a model file generated by train.py. A text file that you want to translate is given to --input. If you need to use GPU, please set the --gpu option.

python translate.py \
  --model ./checkpoints/checkpoint_best.pt \
  --input ./data/sample_test.txt \
  --gpu

• Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems 2017, pp. 5998-6008.