AssertionError: backward pass is invalid for module in evaluation mode about openrlhf HOT 9 CLOSED

Hi, we currently have no free people to help debug it~ if you are interested, you can try it, but we will support QLora in the future~ thx. I guess it may be because of the compatibility of Zero3 and PEFT。

from openrlhf.

This week we'll try to figure it out

from openrlhf.

test with the latest code, please.
This issue is related to the model being incorrectly set to eval mode and not being reverted back to train mode during the backward operation. It should be fixed in the latest code.

from openrlhf.

feel free to reopen it~

from openrlhf.

when i try this code stage2 it work! but i don't know why stage3 is not work,,,

from openrlhf.

thank you for reply

from openrlhf.

thank you and sorry for late response. i will check this tomorrow ! thank you!!!!

from openrlhf.

i run this code and it work!! thank you!!!

from openrlhf.

nice!

from openrlhf.