Sorry I still unkown the whole Train-Test workflow after reading all instructions. about codesearchnet HOT 3 CLOSED

Exactly, the evaluation on the small human-annotated dataset can only be performed via the leaderboard. This is because the dataset is relatively small and it will be easy to overfit on it.

Having said that, you can use the documentation<->code as a proxy task as well as any other additional data you'd like (e.g. if you have some other data that you think can be useful during training/validation).

from codesearchnet.

Hi @Sewens
Does the response here reply to your comment maybe?

from codesearchnet.

I see. You means every query<-->code snippet relevance label can be seen as relationship (pair) between code documentation<-->main code body.

U means for model offline train/test we can only use code you provided. The ture label and negative sample need We need to train a model that can select true code which match the documentation. The annotation will NOT be published, we can only evaluate model via WandB platform.

from codesearchnet.