Questions about the experiments details about lora HOT 4 CLOSED

Thanks for your questions.

1. I believe these are validation numbers since the test set is not public. That is also done by prior work.
2. Nope, that's not a typo. You can verify it with our checkpoint :)

from lora.

Wow, thanks for your quick response. I got two more questions.

1. If I understand correctly, in table 2, the numbers of BitFit were taken from the original paper. But actually, there are some numbers I can not find in the original paper. For example, you mentioned the RoBbase (BitFit) on MRPC task results in 92.7, but I think the original paper reported this number as 92.0 in their Table 2. Could you specify more details about this?
2. Do you fine-tune the bias terms? cause, I understand you don't require gradients for the weight terms, but I did not see you turn off this for bias terms.

from lora.

You are right. I can't remember where we got 92.7, and it should be 92.0.

Yes, the bias term is learnable here, even though it is not in the code used in our experiments. This seems to be a good idea in practice and has a minimal overhead. The checkpointing utility functions should take care of saving/loading biases. Please let me know if you encounter any issues :)

from lora.

Thanks~

from lora.