Does fine-tuned model use the same vocabulary as pre-trained model? about mrasp HOT 5 CLOSED

yes

from mrasp.

appreciate

from mrasp.

If I want to fine-tune this model to support new languages, new tokens should be added to existing file?
That is, new file has larger vocabulary size than 64867?
If do so, the embedding params of checkpoint (e.g. pretrain_checkpoint_last_RAS.pt) released in this project should be expanded before fine-tuning.

from mrasp.

If I want to fine-tune this model to support new languages, new tokens should be added to existing file?
That is, new file has larger vocabulary size than 64867?
If do so, the embedding params of checkpoint (e.g. pretrain_checkpoint_last_RAS.pt) released in this project should be expanded before fine-tuning.

Yes, you are right, if you want to expand the supported languages, you need to merge the newly added tokens into the existing vocabulary, then randomly initialize the embedding vectors of the new tokens.

from mrasp.

appreciate * 2

from mrasp.