This repository contains code base of three separate github repositories. We have only provided sample data and the some of the models such as BERT, BART, RoBERTa to show case how the model works.

Particularly this repository contains codes and dataset access instructions for the EMNLP 2020 publication on understanding empathy expressed in text-based mental health support.

If this code or dataset helps you in your research, please cite the following publication:

@inproceedings{sharma2020empathy,
    title={A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support},
    author={Sharma, Ashish and Miner, Adam S and Atkins, David C and Althoff, Tim},
    year={2020},
    booktitle={EMNLP}
}

We present a computational approach to understanding how empathy is expressed in online mental health platforms. We develop a novel unifying theoretically-grounded framework for characterizing the communication of empathy in text-based conversations. We collect and share a corpus of 10k (post, response) pairs annotated using this empathy framework with supporting evidence for annotations (rationales). We develop a multi-task RoBERTa-based bi-encoder model for identifying empathy in conversations and extracting rationales underlying its predictions. Experiments demonstrate that our approach can effectively identify empathic conversations. We further apply this model to analyze 235k mental health interactions and show that users do not self-learn empathy over time, revealing opportunities for empathy training and feedback.

For a quick overview, check out bdata.uw.edu/empathy. For a detailed description of our work, please read our EMNLP 2020 publication.

Our framework can be compiled on Python 3 environments. The modules used in our code can be installed using:

$ pip install -r requirements.txt

A sample raw input data file is available in dataset/sample_input_ER.csv. This file (and other raw input files in the dataset folder) can be converted into a format that is recognized by the model using with following command:

$ python3 src/process_data.py --input_path dataset/sample_input_ER.csv --output_path dataset/sample_input_model_ER.csv

For training our model on the sample input data, run the following command:

$ python3 src/train.py \
	--train_path=dataset/sample_input_model_ER.csv \
	--lr=2e-5 \
	--batch_size=32 \
	--lambda_EI=1.0 \
	--lambda_RE=0.5 \
	--save_model \
	--save_model_path=output/sample_ER.pth

Note: You may need to create an output folder in the main directory before running this command.

For testing our model on the sample test input, run the following command:

$ python3 src/test.py \
	--input_path dataset/sample_test_input.csv \
	--output_path dataset/sample_test_output.csv \
	--ER_model_path output/sample_ER.pth \
	--IP_model_path output/sample_IP.pth \
	--EX_model_path output/sample_EX.pth

The training script accepts the following arguments:

The Reddit portion of our collected dataset is available inside the dataset folder. The csv files with annotations on the three empathy communication mechanisms are emotional-reactions-reddit.csv, interpretations-reddit.csv, and explorations-reddit.csv. Each csv file contains six columns:

sp_id: Seeker post identifier
rp_id: Response post identifier
seeker_post: A support seeking post from an online user
response_post: A response/reply posted in response to the seeker_post
level: Empathy level of the response_post in the context of the seeker_post
rationales: Portions of the response_post that are supporting evidences or rationales for the identified empathy level. Multiple portions are delimited by '|'

For accessing the TalkLife portion of our dataset for non-commercial use, please contact the TalkLife team here.

Path for output model needs be made for the models

mkdir output

N.B: All three models needs to be generated in order to get the results. We are also providing Testing sample on how to test it out

In order to run the scripting file train.sh/ test.sh, user needs to change the name of the file everytime.