Hi @wangz10,

Thank you for putting together this repository and also the blog post. After going through the code, I had the following doubts. If you could help me clarify I would really appreciate it:

• While training the baseline classifier, did you train the network without normalizing the features? In typical image classification models, the normalization step is not often seen I would say.
• Did you normalize the features for both the encoder and the projector?
• If you could provide some high-level explanation of the max_margin_contrastive_loss function, it would be very helpful.

Thank you and looking forward to hearing from you.

Thanks for your sharing. I have some trouble in logits_mask.
`

    # mask-out self-contrast cases
    logits_mask = torch.scatter(
        torch.ones_like(mask),
        1,
        torch.arange(batch_size * anchor_count).view(-1, 1).to(device),
        0
    )

    mask = mask * logits_mask

    # compute log_prob
    exp_logits = torch.exp(logits) * logits_mask
    #
    log_prob = logits - torch.log(exp_logits.sum(1, keepdim=True))

`
logits_mask is used to filter out the negative pairs, why not the ~mask but one matrix with the diagonals 0 and the others are 1 ?

Hi ,
Thanks for this implimentation . i like to know how to monitor stage 1 training . i am applying this concept for a custom data for image classification. i the visualized projection are not getting grouped together even after running for 50-60 epoch . can you share some idea on how to monitor. also my batch size id limited to 12 because of GPU limitation . do you think this is am issue ?

thanks

Hi, it is strange when using custom dataset the loss is nan (I just tried supervised_nt_xent_loss and max_margin_contrastive_loss). do you have any idea ?

thanks

Why do need base_temperature when calculating the loss?