OpenNMT-py: Open-Source Neural Machine Translation

OpenNMT is an open-source (MIT) neural machine translation system which has 3 different implementations.

The genuine one was a Lua version based on the Harvard Seq2Seq framework. OpenNMT-Lua

The Pytorch version is this repo.

The tensorflow version: OpenNMT-tf

OpenNMT-py is designed to be research friendly to try out new ideas in translation, summary, image-to-text, morphology, and many other domains but also ready for production with a full REST API.

Codebase is relatively stable, but PyTorch is still evolving. We currently recommend forking if you need to have stable code.

OpenNMT-py is run as a collaborative open-source project. The original code was written by Adam Lerer (NYC) and Bryan McCann. Major contributions have come from Sasha Rush and his group (Cambridge, MA), Ben Peters (Saarbrücken), Jianyu Zhan (Shenzhen), Paul Tardy , Vincent Nguyen and many others.

We love contributions. Please consult the Issues page for any Contributions Welcome tagged post.

Table of Contents

• Full Documentation
• Requirements
• Features
• Quickstart
• Citation

Requirements

python 3, torch >=0.4.0, torchtext >=0.2.3, six, tqdm, future, cupy pynvrtc for SRU

pip install -r requirements.txt

Features

The following OpenNMT features are implemented:

• data preprocessing
• Inference (translation) with batching and beam search
• Multiple source and target RNN (lstm/gru) types and attention (dotprod/mlp) types
• TensorBoard logging
• Source word features
• Pretrained Embeddings
• Copy and Coverage Attention
• Image-to-text processing
• Speech-to-text processing
• "Attention is all you need"
• Inference time loss functions.

Beta Features (committed):

• multi-GPU (based on torch distributed, single node at the moment)
• Structured attention
• [Conv2Conv convolution model]
• SRU "RNNs faster than CNN" paper

Quickstart

Full Documentation

Step 1: Preprocess the data

python preprocess.py -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/demo

We will be working with some example data in data/ folder.

The data consists of parallel source (src) and target (tgt) data containing one sentence per line with tokens separated by a space:

• src-train.txt
• tgt-train.txt
• src-val.txt
• tgt-val.txt

Validation files are required and used to evaluate the convergence of the training. It usually contains no more than 5000 sentences.

After running the preprocessing, the following files are generated:

• demo.train.pt: serialized PyTorch file containing training data
• demo.valid.pt: serialized PyTorch file containing validation data
• demo.vocab.pt: serialized PyTorch file containing vocabulary data

Internally the system never touches the words themselves, but uses these indices.

Step 2: Train the model

python train.py -data data/demo -save_model demo-model

The main train command is quite simple. Minimally it takes a data file and a save file. This will run the default model, which consists of a 2-layer LSTM with 500 hidden units on both the encoder/decoder. You can also add -gpuid 1 to use (say) GPU 1.

Step 3: Translate

python translate.py -model demo-model_acc_XX.XX_ppl_XXX.XX_eX.pt -src data/src-test.txt -output pred.txt -replace_unk -verbose

Now you have a model which you can use to predict on new data. We do this by running beam search. This will output predictions into pred.txt.

!!! note "Note" The predictions are going to be quite terrible, as the demo dataset is small. Try running on some larger datasets! For example you can download millions of parallel sentences for translation or summarization.

Pretrained embeddings (e.g. GloVe)

Go to tutorial: How to use GloVe pre-trained embeddings in OpenNMT-py

Pretrained Models

The following pretrained models can be downloaded and used with translate.py.

http://opennmt.net/Models-py/

Citation

OpenNMT technical report

@inproceedings{opennmt,
  author    = {Guillaume Klein and
               Yoon Kim and
               Yuntian Deng and
               Jean Senellart and
               Alexander M. Rush},
  title     = {OpenNMT: Open-Source Toolkit for Neural Machine Translation},
  booktitle = {Proc. ACL},
  year      = {2017},
  url       = {https://doi.org/10.18653/v1/P17-4012},
  doi       = {10.18653/v1/P17-4012}
}