KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning
This is the official code base for the models in our paper on generative commonsense reasoning:
Ye Liu, Yao Wan, Lifang He, Hao Peng, Philip S. Yu. KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning. In AAAI 2021. (https://arxiv.org/abs/2009.12677) This code base is designed to reproduce our experiments in our paper, and can be used with other datasets. For any technical details, please refer to our paper. When using our code or the methods, please cite our paper: reference.
Option 1: Download our processed data:
Option 2: Following our guidness and process from the original data: Download the following data to dataset direction:
- Download the Conceptnet
- Download the CommonGen Dataset, which require to fill the form
python reorder_src.py --dataset_dir "dataset" --save_dataset_dir "dataset/save_dataset"
--org_conceptnet "conceptnet-assertions-5.7.0.csv" --save_conceptnet "conceptnet.csv"
python conceptnet.py --dataset_dir "dataset" --save_dataset_dir "dataset/save_dataset"
--OpenKE_dir "../OpenKE/benchmarks/CommonGen" --save_conceptnet "conceptnet.csv"
cd OpenKE
git clone -b OpenKE-PyTorch https://github.com/thunlp/OpenKE
cd benchmarks/CommonGen
python n-n.py
python train_transe_CommonGen.py
cd ../KG_grounding
python entity_onehot.py --dataset_dir "dataset" --save_dataset_dir "dataset/save_dataset"
--OpenKE_dir "../OpenKE/benchmarks/CommonGen" --save_conceptnet "conceptnet.csv"
Option 2.0.1 The following command shows how to pretrain our KG-BART model with the Conceptnet dataset created by ./get_data.sh as described above:
python pretrain_kgbart.py --data_dir ../dataset/commongen_data/commongen --output_dir ../output/Pretraining_KG
--log_dir ../log/Pretraining_KG2 --pretraining_KG --train_pretraining_num 400000 --val_pretraining_num 80000
--fp16 True --max_seq_length 32 --max_position_embeddings 64 --max_len_a 32 --max_len_b 64 --max_pred 64
--train_batch_size 60 --eval_batch_size 48 --gradient_accumulation_steps 6 --learning_rate 0.00001
--warmup_proportion 0.1 --label_smoothing 0.1 --num_train_epochs 10
Option 2.0.2 Download the Pre-training Weight
The following command shows how to fine-tune our KG-BART model with the CommonGen dataset created by ./get_data.sh as described above:
python run_seq2seq.py --data_dir ../dataset/commongen_data/commongen --output_dir ../output/KGBart
--log_dir ../log/KGBart --model_recover_path ../output/Pretraining_KG/best_model/model.best.bin --fp16 True
--max_seq_length 32 --max_position_embeddings 64 --max_len_a 32 --max_len_b 64 --max_pred 64
--train_batch_size 60 --eval_batch_size 48 --gradient_accumulation_steps 6 --learning_rate 0.00001
--warmup_proportion 0.1 --label_smoothing 0.1 --num_train_epochs 10
Finally, we can evalaute the models on the test set, by running the following command:
python decode_seq2seq.py --data_dir ../dataset/commongen_data/commongen --model_recover_path ../output/KGBart/best_model/model.best.bin
--input_file ../dataset/commongen_data/commongen/commongen.dev.src_alpha.txt --output_dir ../output/KGBart/best_model/Gen
--output_file model.best --split dev --beam_size 5 --forbid_duplicate_ngrams True
Download the Evaluation Package and follow the README: Evaluation
- Python 3 (tested on python 3.6)
- PyTorch (with GPU and CUDA enabled installation)
- amp
- tqdm
Our code is based on Huggingface Transformer. We thank the authors for their wonderful open-source efforts.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent