Code Monkey home page Code Monkey logo

dualug's Introduction

Dual Learning of NLU and NLG

The implementation of the papers "Dual supervised learning for natural language understanding and generation" (ACL 2019) and "Towards Unsupervised Language Understanding and Generation by Joint Dual Learning" (ACL 2020)

Requirements

  • Python >= 3.6
  • torch >= 0.4.1
  • Other required packages are listed in requirements.txt

Setup

# Get the E2ENLG dataset (from the link below), and put it under data/E2ENLG/
$ mkdir -p data/E2ENLG/

# use virtualenv or anaconda to create a virtual environment
# install required packages in requirements.txt

# download SpaCy model
$ python3 -m spacy download en

Usage

# Iterative baseline
python3 main.py \
    --data_dir ../data/ --dataset E2ENLG \
    --fold_attr 1 --vocab_size 500 \
    --regen 0 --replace_model 0 --is_spacy 1 --is_lemma 1 \
    --use_punct 0 --en_max_length -1 --de_max_length -1 \
    --min_length 5 --cell GRU --n_layers 1 --hidden_size 200 \
    --embedding_dim 50  --bidirectional 1  --epochs 10 --batch_size 64 \
    --optimizer Adam --learning_rate 0.001 --teacher_forcing_ratio 0.9  \
    --tf_decay_rate 1.0 --padding_loss 0.0 --eos_loss 1.0 --max_norm 0.25 \
    --verbose_level 1 --save_epochs 1 --is_load 0 --train 1 \
    --dir_name iterative \
    --primal_supervised 1 --dual_supervised 1 \
    --schedule iterative --supervised 1 --model nlu-nlg

# Dual Supervised Learning
python3 run_dsl.py \
    --data_dir ../data/ --dataset E2ENLG \
    --fold_attr 1 --vocab_size 500 \
    --regen 0 --replace_model 0 --is_spacy 1 --is_lemma 1 \
    --use_punct 0 --en_max_length -1 --de_max_length -1 \
    --min_length 5 --cell GRU --n_layers 1 --hidden_size 200 \
    --embedding_dim 50  --bidirectional 1  --epochs 10 --batch_size 64 \
    --optimizer Adam --learning_rate 0.001 --teacher_forcing_ratio 0.9  \
    --tf_decay_rate 1.0 --padding_loss 0.0 --eos_loss 1.0 --max_norm 0.25 \
    --verbose_level 1 --save_epochs 1 --is_load 0 --train 1 \
    --dir_name dsl \
    --model nlu-nlg

# Joint Dual Learning
python3 main.py \
    --data_dir ../data/ --dataset E2ENLG \
    --fold_attr 1 --vocab_size 500 \
    --regen 0 --replace_model 0 --is_spacy 1 --is_lemma 1 \
    --use_punct 0 --en_max_length -1 --de_max_length -1 \
    --min_length 5 --cell GRU --n_layers 1 --hidden_size 200 \
    --embedding_dim 50  --bidirectional 1  --epochs 10 --batch_size 64 \
    --optimizer Adam --learning_rate 0.001 --teacher_forcing_ratio 0.9  \
    --tf_decay_rate 1.0 --padding_loss 0.0 --eos_loss 1.0 --max_norm 0.25 \
    --verbose_level 1 --save_epochs 1 --is_load 0 --train 1 \
    --dir_name joint-nlu_nost-nlg_nost --nlu_st 0 --nlg_st 0 \
    --primal_supervised 1 --dual_supervised 1 \
    --schedule joint --supervised 1 --model nlu-nlg

Please refer to argument.py and example_train.sh for running more experiments. Set --load 1 --train 0 for each command for testing.

References

Main papers to be cited:

@inproceedings{su2019dual,
  title={Dual supervised learning for natural language understanding and generation},
  author={Su, Shang-Yu and Huang, Chao-Wei and Chen, Yun-Nung},
 booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
    year={2019}
}

@inproceedings{su2020joint,
  title={Towards Unsupervised Language Understanding and Generation by Joint Dual Learning},
  author={Su, Shang-Yu and Huang, Chao-Wei and Chen, Yun-Nung},
 booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
    year={2020}
}

Resources

dualug's People

Contributors

shangyusu avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.