How to adapt to new pre-trained models? about ilm HOT 3 CLOSED

If I want to use BART for multi-level text infilling fine-tuning. Which part of the code should I check and revise? Can you provide some hints?

from ilm.

The main thing you'll have to modify is the tokenizer_util.py file. You'll also have to do some modification to the training script.

How does BART tokenize? This modification process will be much easier if it's trivial to align BART tokens to characters. Doing so is not super straightforward with GPT-2 (hence the large amount of code) but may be easier for BART.

from ilm.

The main thing you'll have to modify is the tokenizer_util.py file. You'll also have to do some modification to the training script.

Thanks. I think this piece of information is very helpful ;)

How does BART tokenize? This modification process will be much easier if it's trivial to align BART tokens to characters. Doing so is not super straightforward with GPT-2 (hence the large amount of code) but may be easier for BART.

I think the BART tokenizer won't differ much from GPT-2 and it is built based on the GPT-2 library.

from ilm.