the-ai-summer / byol-cifar10 Goto Github PK

Hi thanks for the amazing explanation of the implementation. However I failed to get the reported accuracy reported in the BLOG. With batch Norm I get an accuracy of around 36% which is also visible in the Notebook you have posted.

But on the second block. The accuracy becomes 70% somehow. Please Let me know if I am missing something.

why I find this error when pretrain

Hi, first of all thanks for writing this code but I saw a problem with your implementation.

In order to make loss symmetric shouldn't the loss be defined in the following way :

loss_one = loss_fn(student_pred_one, teacher_proj_two)
loss_two = loss_fn(student_pred_two, teacher_proj_one)

In your implementation you are computing the loss between the same instance but passed from 2 different branches.
Correct me if I am wrong.