About the question in FAQ: How can I use TrackIn to identify mislabelled examples? about tracin HOT 3 OPEN

One of the primary reasons you see the same list of opponents/proponents for different test examples is that the mislabeled/atypical training instance have gradient magnitudes that are significantly larger than other training instances.

Thinking about this intuitively, two largely orthogonal components determine the magnitude of the dot product you show. First is the alignment of the gradient vectors. The second is the (training) gradient magnitudes. Either or both can cause the dot product to have a large magnitude.

This paper discusses this problem extensively and proposes two different fixes (disclaimer: I am one of the authors). The simple fix is to normalize by the training and test gradient magnitudes. We term this modified version of TracIn GAS.

If you have any questions, you can let me know.

from tracin.

Similar experience. I obtained similar lists of opponents for different test examples. They visually don't make sense.

from tracin.

Got it. Thanks!

from tracin.