thu-keg / copen Goto Github PK

Hi, I'm quite interested in your work and would like to try your code.
Your code is excellent. However, there is several problems that I have to fix before successfully running it.

1. Suggest adding mkdir task[1-3]/data in preprocessing
2. According to README, the data is put in task[1-3]/data. However, in most of the py files for preprocessing and running, the path is task[1-3]/data/ood/
3. For data preprocessing of task2 probing, there is no such file given named data_processor_for_ppl.py. I tried using probing_data_processor.py instead.

Hello, I would like to try my models in the probing settings.
However, it seems that the test sets are not labelled, so we can only submit to codalab.
I spent several hours studying how to generate a file to be submitted to codalab, and now I suppose files like answer-bert-base-uncased-probs.json are what we should submit. Is it right?
Also, I found that there are an argument named "mask_position", which is a value from [all, answer, concept], and these values yield corresponding files. I wonder what's the difference among these values.