Optimizer settings for DONUT pre-training on Synthdog about donut HOT 3 CLOSED

Yes, it was 1 sample per V100, and the normalized edit distance for the dev set was 0.0045.

from donut.

Hi @Veason-silverbullet ,

The architecture settings are available at config.json and more information is available in the manuscript (https://arxiv.org/abs/2111.15664), but I believe you have already checked it :) donut-proto shares most settings with donut-base, but it used 8e-5 as a constant rate. I wonder if your model training converged or not. Note that the donut-proto is trained with 5 GPU days.
Plus, note that donut-proto used "<s_synthdog>" as a task start token for all SynthDoG-generated data, as it can be checked at https://huggingface.co/naver-clova-ix/donut-proto/blob/official/added_tokens.json#L1. You can manually set the token with:

• https://github.com/clovaai/donut/blob/1.0.7/train.py#L83-L85

Hope this helps :) Please let me know if you are still confused. Feel free to reopen this or open another issue if you have anything new for sharing.

from donut.

Hi @gwkrsrch ,
Yes, the model I trained may not converge. May I know the batch size and final training loss of DONUT-proto pretraining, so that I can know if my experimental model converged?

from donut.