Issue with evaluating model with beam search about kobe HOT 4 CLOSED

Hi @pavlos2094 ,

Thanks for your interest in this work. This problem is caused by how division is treated in different versions of PyTorch (line prevK = bestScoresId / numWords). Please either use the torch version in README or modify this line to prevK = bestScoresId // numWords

from kobe.

@qibinc thank you for your immediate response!

from kobe.

@qibinc i follow your suggestion with "modify the line prevK = bestScoresId // numWords", but it seems still something wrong. The progress bar does not change for maintaining 0% and can not get the evaluating result
0%| | 0/82 [00:00<?, ?it/s]

from kobe.

Hi @ssexuejinwei ,

We have rewritten this codebase with latest versions of DL frameworks and significantly improved the installation, code quality, reproducibility and visualization.

We have also replaced beam search with nucleus sampling, a method proven to be more suitable in text generation than beam search to avoid repetition. You can find how to use it at https://github.com/THUDM/KOBE#evaluating-kobe.

We also added BERTScore for evaluation, a metric better than BLEU in aligning with human judgement.

Here is a screenshot of the KOBE-v2's training progress, in case your are still interested:

Best,

from kobe.