R-ROME RuntimeError: expected scalar type Half but found Float about easyedit HOT 10 CLOSED

I suggest you set fp16 to false, as setting it to true often results in unexpected outcomes.

from easyedit.

Sure thanks. One more thing. Is the R-ROME implementation supporting llama2-7b and gpt-j as in hparams or basically all the models as stated in README?

from easyedit.

Yes, all are supported. You can just modify the parameters a bit.

from easyedit.

Thnaks a lot! Now I changed fp16 to false. I'm exceeding 50gb of memory (2 x RTX 4090) for Mistral-7b. It barely worked on llama2-7b taking over 46gb. Should it be as memory consuming to run ROME?

from easyedit.

Same happened for KN model. Just running out of memory with 50gb in paralell setup on llama-7b.

from easyedit.

I don't think there's any problem; this is the normal memory usage. You can try setting it to fp16 at

from easyedit.

Hi, do you have any further questions?

from easyedit.

Hi, thank you for your answer. Yes, maybe one more.
How exactly in current implementation for example when I use editor with ROME locality, probability, rewrite and rephrase accuracies are calculated?

from easyedit.

You can check the file at easyeditor/evaluate/evaluate_utils.py. Except for locality, we calculate the number of matching tokens and then take the average for everything else. Locality calculates whether unrelated inputs have changed before and after the edits.

from easyedit.

Ok, thank you very much :)

from easyedit.