Questions about the evaluation of MIA efficacy about unlearn-sparse HOT 1 CLOSED

Thanks for making the code public.

I have a question on this line of code:

Unlearn-Sparse/evaluation/SVC_MIA.py

Line 67 in 76a4299

acc_train = clf.predict(X_target_train).mean()

Wouldn't the accuracy be calculated by comparing the predictions with the ground truth? Did I missing something.

Thanks for the good question! Here we're doing binary classification. Thus, we split the positive (L65-68) and negative (L70-73) classes and get an average to balance the impact of the two classes when they have different data points.

In our paper, Appendix C.3, clarifies the implementation of our MIA metric. Here we count the true negatives predicted by our MIA predictor, i.e. how much data in the forgetting set would be classified as a training example. So only the positive one has been activated here.

Please feel free to let me know if you have further questions!

from unlearn-sparse.