mayer123 / hykas-cskg Goto Github PK

Hello.
During reproducing your code, I have some issues and questions.

To reproduce your code, I set up the environment as same as yours except pytorch due to the GPU device:
→ Python 3.7.6, Pytorch 1.10.0+cu113, and Transformers 3.0.2

1. I got the lower accuracy for RoBERTa-L(MR) which is trained with the ATOMIC dataset that you uploaded in Github as below:

   • CSQA: 62.4 (64.2 ( ± 0.7)
 in paper)
   • SIQA: 61.7 (63.1 ( ± 1.5) in paper)
   • WG: 53.8 (59.6 ( ± 0.3)
 in paper)

   For aNLI and PIQA, I got the similarly accuracy as the performance in your paper.

   • aNLI: 69.8
   • PIQA: 71.8

   I trained the model with the hyper-parameters that you shared in Github, which is the same as the hyper-parameters in your paper. Are there any parameters should I change?

2. I got the warning below:

   [W Resize.cpp:23] Warning: An output with one or more elements was resized since it had shape [], which does not match the required output shape [2, 3].This behavior is deprecated, and in a future PyTorch release outputs will not be resized unless they have zero elements. You can explicitly reuse an out tensor t by resizing it, inplace, to zero elements with t.resize_(0). (function resize_output_check)

   Is there any way to hide this warning only? or to solve this issue?

Any reply would be greatly appreciated.