The implementation of the papers "Dual supervised learning for natural language understanding and generation" (ACL 2019) and "Towards Unsupervised Language Understanding and Generation by Joint Dual Learning" (ACL 2020)
- Python >= 3.6
- torch >= 0.4.1
- Other required packages are listed in
requirements.txt
# Get the E2ENLG dataset (from the link below), and put it under data/E2ENLG/
$ mkdir -p data/E2ENLG/
# use virtualenv or anaconda to create a virtual environment
# install required packages in requirements.txt
# download SpaCy model
$ python3 -m spacy download en
# Iterative baseline
python3 main.py \
--data_dir ../data/ --dataset E2ENLG \
--fold_attr 1 --vocab_size 500 \
--regen 0 --replace_model 0 --is_spacy 1 --is_lemma 1 \
--use_punct 0 --en_max_length -1 --de_max_length -1 \
--min_length 5 --cell GRU --n_layers 1 --hidden_size 200 \
--embedding_dim 50 --bidirectional 1 --epochs 10 --batch_size 64 \
--optimizer Adam --learning_rate 0.001 --teacher_forcing_ratio 0.9 \
--tf_decay_rate 1.0 --padding_loss 0.0 --eos_loss 1.0 --max_norm 0.25 \
--verbose_level 1 --save_epochs 1 --is_load 0 --train 1 \
--dir_name iterative \
--primal_supervised 1 --dual_supervised 1 \
--schedule iterative --supervised 1 --model nlu-nlg
# Dual Supervised Learning
python3 run_dsl.py \
--data_dir ../data/ --dataset E2ENLG \
--fold_attr 1 --vocab_size 500 \
--regen 0 --replace_model 0 --is_spacy 1 --is_lemma 1 \
--use_punct 0 --en_max_length -1 --de_max_length -1 \
--min_length 5 --cell GRU --n_layers 1 --hidden_size 200 \
--embedding_dim 50 --bidirectional 1 --epochs 10 --batch_size 64 \
--optimizer Adam --learning_rate 0.001 --teacher_forcing_ratio 0.9 \
--tf_decay_rate 1.0 --padding_loss 0.0 --eos_loss 1.0 --max_norm 0.25 \
--verbose_level 1 --save_epochs 1 --is_load 0 --train 1 \
--dir_name dsl \
--model nlu-nlg
# Joint Dual Learning
python3 main.py \
--data_dir ../data/ --dataset E2ENLG \
--fold_attr 1 --vocab_size 500 \
--regen 0 --replace_model 0 --is_spacy 1 --is_lemma 1 \
--use_punct 0 --en_max_length -1 --de_max_length -1 \
--min_length 5 --cell GRU --n_layers 1 --hidden_size 200 \
--embedding_dim 50 --bidirectional 1 --epochs 10 --batch_size 64 \
--optimizer Adam --learning_rate 0.001 --teacher_forcing_ratio 0.9 \
--tf_decay_rate 1.0 --padding_loss 0.0 --eos_loss 1.0 --max_norm 0.25 \
--verbose_level 1 --save_epochs 1 --is_load 0 --train 1 \
--dir_name joint-nlu_nost-nlg_nost --nlu_st 0 --nlg_st 0 \
--primal_supervised 1 --dual_supervised 1 \
--schedule joint --supervised 1 --model nlu-nlg
Please refer to argument.py
and example_train.sh
for running more experiments. Set --load 1 --train 0
for each command for testing.
Main papers to be cited:
@inproceedings{su2019dual,
title={Dual supervised learning for natural language understanding and generation},
author={Su, Shang-Yu and Huang, Chao-Wei and Chen, Yun-Nung},
booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
year={2019}
}
@inproceedings{su2020joint,
title={Towards Unsupervised Language Understanding and Generation by Joint Dual Learning},
author={Su, Shang-Yu and Huang, Chao-Wei and Chen, Yun-Nung},
booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
year={2020}
}