Question about the lambda about bk-sdm HOT 2 CLOSED

Thanks for the information.

from bk-sdm.

Hi, thanks for your inquiry.

For our text-to-image experiments, we simply set the loss weights λ_Task, λ_OutKD, and λ_FeatKD to 1, which was effective in empirical validation without hyperparameter tuning and was used in the experiments of our paper.

In recent trials with BK-SDM-Small and batch size 64, changing λ_FeatKD to {0.25, 0.5, 1, 2, 4} did not affect the final generation scores. However, using different scales like 0.01, 0.1, 10, or 100 hasn't been explored.

It would be interesting to study the effect of different loss weightings.

added: some experimental results were as follows:

• recent trials with BK-SDM-Small, batch size 64, changing λ_FeatKD to {0.25, 0.5, 1, 2, 4}
  

• the ablation study presented in our paper (v: loss weight = 1, x: loss weight = 0)
  

from bk-sdm.