How to regulate the alpha and the beta of SCEloss? about symmetric_cross_entropy_for_noisy_labels HOT 4 CLOSED

Hi firelgh,

1. If the labels/mask are clean, try 1*CE + beta*RCE, beta can be selected to make sure the two loss terms have similar magnitudes. In terms of segmentation, it is related to the sparsity of the mask. Sometimes, the value of beta*RCE can be 10x large than the CE term, which may produce better performance.

2. If the labels/mask are noisy, try around 0.1*CE + beta*RCE. The 0.1*CE is to reduce the overfitting of CE to noisy mask. The beta can be large such as 6/10 to boost training.

3. If the dataset is very sparse, and has a convergence problem. The try large alpha [1, 10]. For example: 6*CE+ beta*RCE. The beta can be determined similarly as above.

Note the A parameter can also be tuned. The beta works together with A, and they determine the magnitude of the RCE term. For segmentation, there are usually many dimensions in a mask, try A=-1/-2 instead of the -4 as used for classification.

Hope this can help your experiments.

from symmetric_cross_entropy_for_noisy_labels.

Thank you very much! It was very helpful to me.

However,there's a question about parameter A. In the code, I can‘t find where the parameter A is defined from.

In the code, I only find the alpha and the bete of SCEloss.

def symmetric_cross_entropy(alpha, beta):
def loss(y_true, y_pred):
y_true_1 = y_true
y_pred_1 = y_pred

    y_true_2 = y_true
    y_pred_2 = y_pred

    y_pred_1 = tf.clip_by_value(y_pred_1, 1e-7, 1.0)
    y_true_2 = tf.clip_by_value(y_true_2, 1e-4, 1.0)

    return alpha*tf.reduce_mean(-tf.reduce_sum(y_true_1 * tf.log(y_pred_1), axis = -1)) + beta*tf.reduce_mean(-tf.reduce_sum(y_pred_2 * tf.log(y_true_2), axis = -1))
return loss

from symmetric_cross_entropy_for_noisy_labels.

Hi firelgh,
y_true_2 = tf.clip_by_value(y_true_2, 1e-4, 1.0), this is related to A.

from symmetric_cross_entropy_for_noisy_labels.

Okay,I've got it.Thank you.

from symmetric_cross_entropy_for_noisy_labels.