why keep "###" before instruct text? about video-llama HOT 6 CLOSED

During inference, we will stop generating tokens once the assistant outputs "###".
Suppose that the training data is " ###Human: Hi. ###Assistant: Hi, can I help you. ###Human: yes".
The content which needs to calculate loss is "Hi, can I help you. ###". So the model can learn to generate "###" as the end flag of his reply.

from video-llama.

The assistant will learn to generate "###" if it wants to end the current round. So "###" can be understood as the EOS flag of each session.

from video-llama.

The assistant will learn to generate "###" if it wants to end the current round. So "###" can be understood as the EOS flag of each session.

Thanks for the reply, I understand "###" works like EOS here, and I think the "###" in assistant text is more like EOS? howerver why do we want assistant to learn EOS before instruct?

from video-llama.

During inference, we will stop generating tokens once the assistant outputs "###". Suppose that the training data is " ###Human: Hi. ###Assistant: Hi, can I help you. ###Human: yes". The content which needs to calculate loss is "Hi, can I help you. ###". So the model can learn to generate "###" as the end flag of his reply.

I mean you are also keep the first "###"(the one before "Human: Hi...").

from video-llama.

Oh, I see. This place is just for convenience, and you can add a judgment, mask the first "###".

from video-llama.

Okay, thanks for the confirmation

from video-llama.