loss =87.3 about sphereface HOT 9 OPEN

see https://github.com/wy1iu/LargeMargin_Softmax_Loss#notes-for-training

from sphereface.

For large and difficult datasets, you should first try to set lambda_min as 5 or 10
Is this one?
But, when I use sphereface-20, I can get a small lambda_min and the results is good.

from sphereface.

It really depends on the dataset and the network architecture. There is no universally good hyper-parameter in all cases. You should change it depending on your task, dataset and the network.

from sphereface.

Ok, How to determine lambda_min?
train softmax_loss ? train accuracy? or test results?

from sphereface.

Maybe trying a large value first, and gradually decreasing it could be a good strategy.

Besides the lambda_min. the other hyperparameters also affect the convergence.

from sphereface.

Okay, I down it from 1000, and my boss wants me to drop it to 5.
I'll try it for more numbers!
Thanks again.
Best!

from sphereface.

87.3365 is a magic number. if the data is nan, the softmaxloss will be 87.3365. You can print the debug info to see the wrong. And I think you can print the x_norm and weight norm in the margin_inner product , maybe the x_norm is 0 which will make the data to be nan.

from sphereface.

@fromwhzz I suffered the same problem and according to your suggestion, I found L1 norm = (nan, nan); L2 norm = (nan, nan), loss is always 87.3365. What should I do to solve it? Thanks a lot.

from sphereface.

@wy1iu
For a custom dataset, I have 850 ids for train and 877 ids for test with 909 unique ids.

what number should I use for the margin_inner_product in softmax loss ? Also, I have the same problem with softmaxloss=87.335 and final of net.forward with a known image is nan.

Thanks
Chandra

from sphereface.