Consistency between code and paper about cutmix-pytorch HOT 4 CLOSED

Thanks for the question.
We propose to mix the two targets not using a single target in the paper. As you can see (1) and Appendix A1.

Equation (1)

Appendix A1

Here, each "y" is a one-hot label (thus an example of \hat y is [0.6, 0.4, 0, 0, 0])
Note that computing the cross-entropy between the mixed target and the prediction is equivalent to mix two cross entropies with a single target, i.e.,

CE(mixed_label_between_ya_yb, prediction) = lambda * CE(ya, prediction) + (1 - lambda) CE(yb, prediction)

The above equation is easily derived from the definition of the cross entropy.
There is no change after we made the first version of the CutMix algorithm, and our code is consistent with our paper.
Thanks

from cutmix-pytorch.

I'm greatly appreciated for your prompt response!

I didn't notice the equivalence be satisfied before!

But still wondering, is there a particular issue on implementing with CE(mixed_label_between_ya_yb, prediction)? Why did you implement with the equivalent form, not the original one? At first glance, the original formula is also equally easy to implement.

Thank you!

from cutmix-pytorch.

@shim94kr
Technically speaking, the CE in my comment and CE in PyTorch is not equivalent.
The cross entropy implementation of PyTorch, and most ML frameworks, gets "index" not the probability itself. I.e., "target" is not a one-hot (e.g., [1, 0, 0, 0, ...]) but index value (e.g., 34) for PyTorch CE.
Since the cross entropy implementation is highly optimized ("This criterion combines LogSoftmax and NLLLoss in one single class" from the document), we do not create our own CE implementation.
See the official document for the details.

from cutmix-pytorch.

I'm now clear on this issue. Thank you!

from cutmix-pytorch.