declare-lab / mime Goto Github PK

This repository contains PyTorch implementations of the models from the paper An Empirical Study MIME: MIMicking Emotions for Empathetic Response Generation.

License: MIT License

Python 97.19% Perl 2.81%

dialogue-systems conversational-bots natural-language-processing transformer-encoder empathetic-dialogues empathetic-responses

mime's Introduction

MIME

This repository contains PyTorch implementation of empathetic-response generation model MIME: MIMicking Emotions for Empathetic Response Generation.

Overview of MIME

Current approaches to empathetic response generation view the set of emotions expressed in the input text as a flat structure, where all the emotions are treated uniformly. We argue that empathetic responses often mimic the emotion of the user to a varying degree, depending on its positivity or negativity and content. We show that the consideration of these polarity-based emotion clusters and emotional mimicry results in improved empathy and contextual relevance of the response as compared to the state-of-the-art. Also, we introduce stochasticity into the emotion mixture that yields emotionally more varied empathetic responses than the previous work. We demonstrate the importance of these factors to empathetic response generation using both automatic- and human-based evaluations.

Setup

Download GloVe vectors from here and put it into vectors/ folder

Next Install the required libraries:

Assume you are using conda
Install libraries by pip install -r requirement.txt

For reproducibility purposes, we provide model output on test dataset as ./output.txt and weights in google drive. you can download the model and move it under save/saved_model

Run code

Dataset is already preprocessed and contained in this repo. we used proprocessed data provided in this link https://github.com/HLTCHKUST/MoEL/tree/master/empathetic-dialogue.

Training

python main.py

Note: This will also generate output file on test dataset as save/test/output.txt.

Testing

python main.py --test --save_path [output_file_path]

Note: During testing, the model will load weight under save/saved_model, and by default it will generate save/test/output.txt as output file on test dataset.

Citation

MIME: MIMicking Emotions for Empathetic Response Generation. Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, Soujanya Poria. EMNLP (2020).

mime's People

Contributors

Stargazers

Watchers

Forkers

emrys-hong xl2248 soujanyaporia jsedoc laibabasit008 suhasagg tinaingalaxy ajyl lxh6636 eyrechan cooperleong00

mime's Issues

save/saved_model

In save/saved_model, there are no model weights in the model path, and there are no models and weights in the link provided in readme. Where should I download them?Thank you

Question about the output.txt

Hi,
I found that the responses provided in output.txt are not similar to what you mentioned in your paper.

E.g.,
emotion:apprehensive vader-score: -0.6606 predicted_emotion: afraid
Context:['i went skydiving last summer with my partner . it was so scary !', 'wow , that is scary . were you nervous ?', 'absolutely ! i have always been scared of heights , but i am so glad i made myself .']
Topk:that sounds like fun !
Beam: i am sorry to hear that . i hope you have a good time !
Greedy:i am sorry to hear that . i hope you get it !
Ref:it sounds like a great experience !

While the response for this case in your paper is "That must've been a very beautiful memory."

How do I test the model on custom input?

I have saved a model after training and loaded it as -

model = torch.load('/home/ayan/Data1/MIME/test_model.pt')
model.eval()

And the model is tested by the following code

model.train_one_batch(batch, 0, train = False)

This returns four metrics, viz loss, ppl, bce_prog and acc_prog, but this is with respect to the output text already provided in batch as test_context, test_situation, test_emotion, and test_target.

Can you please tell me how do I get the original response of the model as output for an input that I provide which is not in ./dataset/*.

Can't download the weights of the model

I want to evaluate the model (I downloaded weights from google disk) , however unfortunately during testing the following error occurs:

RuntimeError: Error(s) in loading state_dict for Train_MIME:
	Missing key(s) in state_dict: "encoder.enc.multi_head_attention.query_linear.weight", 
"encoder.enc.multi_head_attention.key_linear.weight", "encoder.enc.multi_head_attention.value_linear.weight", 
"encoder.enc.multi_head_attention.output_linear.weight", "encoder.enc.positionwise_feed_forward.layers.0.conv.weight", 

"encoder.enc.positionwise_feed_forward.layers.0.conv.bias", "encoder.enc.positionwise_feed_forward.layers.1.conv.weight",
 "encoder.enc.positionwise_feed_forward.layers.1.conv.bias", "encoder.enc.layer_norm_mha.gamma", 
"encoder.enc.layer_norm_mha.beta", "encoder.enc.layer_norm_ffn.gamma", "encoder.enc.layer_norm_ffn.beta", 
"emotion_input_encoder_1.enc.enc.multi_head_attention.query_linear.weight", 
"emotion_input_encoder_1.enc.enc.multi_head_attention.key_linear.weight", 
"emotion_input_encoder_1.enc.enc.multi_head_attention.value_linear.weight", 
"emotion_input_encoder_1.enc.enc.multi_head_attention.output_linear.weight", 
"emotion_input_encoder_1.enc.enc.positionwise_feed_forward.layers.0.conv.weight", 
"emotion_input_encoder_1.enc.enc.positionwise_feed_forward.layers.0.conv.bias", 
"emotion_input_encoder_1.enc.enc.positionwise_feed_forward.layers.1.conv.weight", 
"emotion_input_encoder_1.enc.enc.positionwise_feed_forward.layers.1.conv.bias", 
"emotion_input_encoder_1.enc.enc.layer_norm_mha.gamma", "emotion_input_encoder_1.enc.enc.layer_norm_mha.beta", 
"emotion_input_encoder_1.enc.enc.layer_norm_ffn.gamma", "emotion_input_encoder_1.enc.enc.layer_norm_ffn.beta", 
"emotion_input_encoder_2.enc.enc.multi_head_attention.query_linear.weight", 
"emotion_input_encoder_2.enc.enc.multi_head_attention.key_linear.weight", 
"emotion_input_encoder_2.enc.enc.multi_head_attention.value_linear.weight", 
"emotion_input_encoder_2.enc.enc.multi_head_attention.output_linear.weight", 
"emotion_input_encoder_2.enc.enc.positionwise_feed_forward.layers.0.conv.weight", 
"emotion_input_encoder_2.enc.enc.positionwise_feed_forward.layers.0.conv.bias", 
"emotion_input_encoder_2.enc.enc.positionwise_feed_forward.layers.1.conv.weight", 
"emotion_input_encoder_2.enc.enc.positionwise_feed_forward.layers.1.conv.bias", 
"emotion_input_encoder_2.enc.enc.layer_norm_mha.gamma", "emotion_input_encoder_2.enc.enc.layer_norm_mha.beta", 
"emotion_input_encoder_2.enc.enc.layer_norm_ffn.gamma", "emotion_input_encoder_2.enc.enc.layer_norm_ffn.beta". 

	Unexpected key(s) in state_dict: "encoder.enc.0.multi_head_attention.query_linear.weight", 
"encoder.enc.0.multi_head_attention.key_linear.weight", "encoder.enc.0.multi_head_attention.value_linear.weight", 
"encoder.enc.0.multi_head_attention.output_linear.weight", "encoder.enc.0.positionwise_feed_forward.layers.0.conv.weight", 
"encoder.enc.0.positionwise_feed_forward.layers.0.conv.bias", 
"encoder.enc.0.positionwise_feed_forward.layers.1.conv.weight", 
"encoder.enc.0.positionwise_feed_forward.layers.1.conv.bias", "encoder.enc.0.layer_norm_mha.gamma", 
"encoder.enc.0.layer_norm_mha.beta", "encoder.enc.0.layer_norm_ffn.gamma", "encoder.enc.0.layer_norm_ffn.beta", 
"emotion_input_encoder_1.enc.enc.0.multi_head_attention.query_linear.weight", 
"emotion_input_encoder_1.enc.enc.0.multi_head_attention.key_linear.weight", 
"emotion_input_encoder_1.enc.enc.0.multi_head_attention.value_linear.weight", 
"emotion_input_encoder_1.enc.enc.0.multi_head_attention.output_linear.weight", 
"emotion_input_encoder_1.enc.enc.0.positionwise_feed_forward.layers.0.conv.weight", 
"emotion_input_encoder_1.enc.enc.0.positionwise_feed_forward.layers.0.conv.bias", 
"emotion_input_encoder_1.enc.enc.0.positionwise_feed_forward.layers.1.conv.weight", 
"emotion_input_encoder_1.enc.enc.0.positionwise_feed_forward.layers.1.conv.bias", 
"emotion_input_encoder_1.enc.enc.0.layer_norm_mha.gamma", "emotion_input_encoder_1.enc.enc.0.layer_norm_mha.beta", 
"emotion_input_encoder_1.enc.enc.0.layer_norm_ffn.gamma", "emotion_input_encoder_1.enc.enc.0.layer_norm_ffn.beta", 
"emotion_input_encoder_2.enc.enc.0.multi_head_attention.query_linear.weight", 
"emotion_input_encoder_2.enc.enc.0.multi_head_attention.key_linear.weight", 
"emotion_input_encoder_2.enc.enc.0.multi_head_attention.value_linear.weight", 
"emotion_input_encoder_2.enc.enc.0.multi_head_attention.output_linear.weight", 
"emotion_input_encoder_2.enc.enc.0.positionwise_feed_forward.layers.0.conv.weight", 
"emotion_input_encoder_2.enc.enc.0.positionwise_feed_forward.layers.0.conv.bias", 
"emotion_input_encoder_2.enc.enc.0.positionwise_feed_forward.layers.1.conv.weight", 
"emotion_input_encoder_2.enc.enc.0.positionwise_feed_forward.layers.1.conv.bias", 
"emotion_input_encoder_2.enc.enc.0.layer_norm_mha.gamma", "emotion_input_encoder_2.enc.enc.0.layer_norm_mha.beta",
 "emotion_input_encoder_2.enc.enc.0.layer_norm_ffn.gamma", "emotion_input_encoder_2.enc.enc.0.layer_norm_ffn.beta".

I rewrote the saved_model dict in the following way:
enc.0 -> enc
That works for me. Is it correct way for downloading model's weights?

Thank you for your help,
Anastasiia

Best model weights

Hi!
Can you please say which model is the best: saved_model or old/saved_model?
Which one is the model with std and which one without?
Thank you!

Dataset Sample Number Mismatched

Hi, I was trying to evaluate your model with the output.txt. However, I found that the number of the dataset sample was different. There are supposed to have 5259 samples in the testing set, but yours only get 5255 samples.

For instance, the last dialogue (the last 2 turns of sample) should be:

I have a big test on Monday_comma_ I am so nervous.
What is the test on?
It's for my Chemistry class. I haven't slept much because I'm worried so I keep studying.
Chemistry is quite difficult_comma_have you studied hard?

"sys_dialog_texts.test.npy" also get 5255 samples.
Maybe you should check it out, leave a message if you get any news. Thank you.

requirement.txt 在哪里

pip install -r requirement.txt
可是文件里没有 requirement.txt文件，请问在哪里？谢谢

Questions about code reproduction

Hello,
I successfully ran the program according to the README.md, but due to the patience of the program, my program ended early, and retrained in the stage at about 18000 period. is this normal? It seems the model can't learn the data well, then we get a worse result when testing.
When I ran the MIME, I finally got the result:
EVAL Loss PPL Accuracy Bleu_g Bleu_b Blue_t
valid 3.7737 43.5429 0.33 0.00 0.00 0.00
testing generation:
EVAL Loss PPL Accuracy Bleu_g Bleu_b Blue_t
test 3.6466 38.3426 0.33 2.50 2.61 1.57
How can I set up the code to reproduce the results of the paper?