Are the training loss and validation loss recorded? about magicoder HOT 4 CLOSED

Thank you for your open source, I am replicating your fine-tuning process according to the code on github. Do the results of train loss=0.16 and eval_loss=0.21 I trained on the 75k dataset match yours? I will continue training on the 110k dataset.
I trained for 4 epochs and indeed started overfitting after the second epoch.

from magicoder.

Magicoder-S-CL.json
Magicoder-CL.json
Magicoder-S-DS.json
Magicoder-DS.json

Hi, here are the trainer states. Hope they can help!

from magicoder.

Magicoder-S-CL.json Magicoder-CL.json Magicoder-S-DS.json Magicoder-DS.json

Hi, here are the trainer states. Hope they can help!

Thank you very much, my training process is basically the same as your loss metric and the test results on humaneval dataset are basically consistent

But I have a question. Why is the model fully fine-tuned by instruction, but why is the capacity infilinig increased?
I look forward to hearing from you.

from magicoder.

Good to hear you can reproduce it. Yeah we did observe that the infilling capability was at least not decreasing. We believe this is because the model learned some general alignment during the instruction tuning, and infilling is a kind of alignment based on the surrounding context. Further study of this phenomenon would be interesting.

from magicoder.