This is excellent work, and thanks a lot for your open-source code. However, when I reproduced your work, I found that I could not achieve the results reported in the article.
In reproducing, I followed exactly the steps in the readme.md
and used the same hardware. Besides, in the article, you report a learning rate of 2e-4,
whereas the default learning rate in the code is 5e-4
. In reproducing, I found that the result was better when the learning rate was set to 5e-4
.
I don't know what's wrong, and I hope I can get your help. Thank you very much.
The following results are reproduced.
When the learning rate is set to 5e-4
:
DDI: 0.0632 (0.0003) Ja: 0.5114 (0.0026) F1: 0.6676 (0.0023) PRAUC: 0.7649 (0.0028)
When the learning rate is set to 2e-4
:
DDI: 0.0607 (0.0005) Ja: 0.5089 (0.0022) F1: 0.6659 (0.0019) PRAUC: 0.7632 (0.0022)
The results reported in the article:
DDI: 0.0589 (0.0005) Ja: 0.5213 (0.0030) F1: 0.6768 (0.0027) PRAUC: 0.7647 (0.0025)