loss is nan about softtriple HOT 7 CLOSED

Hi, you can obtain the probability over all classes as: prob = F.softmax(self.la*simClass, dim=1). BTW, a larger feature dimension, e.g., 512, usually can improve the performance.

from softtriple.

Hi, besides the lr, you may try to reduce the value of lambda to a small one, e.g., 10, for a smooth optimization.

from softtriple.

Thank you so much for your answer !

I have turned lambda to 10 and center to 3 , it works well !

But in my project (a long tail dataset for classification), I use ResNet-18 as training model and train from scratch . To compare SoftTriple with cross entropy , I fix most parameters . The differences are last fc layer , the output dim of model of SoftTriple changed from class numbers (247) to 64 , and putting the parameters of loss function into optimizer . To my surprise,the accuracy of SoftTriple is a little bit (about 0.5%) smaller than CE !

For me , I think the hyperparameters of loss function need to be fine tuned , do you have any advice on it?

Thank you ! Have a good day !

from softtriple.

Hi, first you can set K=1 to check if it can reproduce the performance of CE with the appropriate setting. After that, you may tune parameters for SoftTriple, e.g., gamma, K, etc. to obtain the additional gain.

from softtriple.

Thank you!

from softtriple.

@qian-qi

Sorry for trouble you again!

When I try to use SoftTriple loss for classification task , you know that for most model the output dim is equal to class number and we can get probabilities of each class. To use SoftTriple loss , I change the dim of model from class number to 64 as in your code . For measuring, I use self.la*(simClass-marginM) in loss function as probabilities of each class.Because I think we usually put output preds and labels into loss function (lossClassify = F.cross_entropy(self.la*(simClass-marginM), target)). I don't know whether it is correct ?

In your validate function,you use the code:
sim = X.dot(X.T)
minval = np.min(sim) - 1.
sim -= np.diag(np.diag(sim))
sim += np.diag(np.ones(num) * minval)
indices = np.argsort(-sim, axis=1)[:, : kmax]
YNN = Y[indices]
can I take YNN as the Top K output class to calculate accuracy ? Can we take YNN[i][0] as the t predict top-1 class? I am a little confused about this code.

from softtriple.

Thank you so much!

from softtriple.