Evaluation codes not found about magicoder HOT 3 CLOSED

Hi, the evaluation code is all under /experiments. We will document the procedure to reproduce the numbers later.

from magicoder.

Thank you very much for the hint!

I have been studying codes in /experiments these days.
I spotted the code description of init.py in /experiments, '[feat: training and humaneval evaluation code]'. I thought I finally found the codes for HumanEval, but nothing inside.
I tried to follow the instructions in README.md , I can't figure out what and where the 'data-clean-decontaminated.jsonl' is.

It will be great if providing more detailed descriptions.

from magicoder.

Here is the code for HumanEval and MBPP evaluation: https://github.com/ise-uiuc/magicoder/blob/main/experiments/text2code.py

It will write results into a jsonl file that has evalplus-compatible format. You can then refer to evalplus documentation to get the pass@1 results.

from magicoder.