Will the the both model, the biased original model and adversarial one return the same the prediction for a test real sample?

Here the y label is set as whether the sample is real or synthesized. With the different target definition, I am confused the both two model return the same prediction values.

Hey! I finally got around to playing with the examples you have here, and I noticed that you were using shap.kmeans to get the background data. Since I typically use a random sample not kmeans (unless I am trying to really trying to play with run time optimization), I just swapped

background_distribution = shap.kmeans(xtrain,10)

for

background_distribution = shap.sample(xtrain,10)

When I did this all the adversarial results for SHAP seemed to fall apart for COMPAS...meaning 79% of the time race is still the top SHAP feature in the test dataset for the adversarial model.

This very strong dependence on using kmeans was surprising to me, since it seems to imply SHAP is much more robust to these adversarial attacks when using a typical random background sample. Have you noticed this before, or do you have any thoughts on this? I think it is worth pointing out, but I wanted to get your feedback before suggesting to users that a random sample provides better adversarial robustness.

Thanks!

I would like to use my own estimator while training the adversary model. So, i do this:
adv_lime = Adversarial_Lime_Model(racist_model_f(), innocuous_model_psi()).\ train(xtrain, ytrain, feature_names=features, categorical_features=categorical_feature_indcs, estimator=knn)
But when i degubbed the training, i noticed that the estimator is still RandomForestClassifier from /Fooling-LIME-SHAP/adversarial_models.py line 177-182. Not sure the reason, then i commented out #self.perturbation_identifier = RandomForestClassifier(n_estimators=rf_estimators).fit(xtrain, ytrain) and used my own estimator by: self.perturbation_identifier = KNeighborsClassifier().fit(xtrain, ytrain). I even commented out line 5: # from sklearn.ensemble import RandomForestClassifier just to see what is going on. the program then uses the KNeighborsClassifier at line 180 but the self.perturbation_identifier on line 182 is None somehow. and for that, i get this error: ['NoneType' object has no attribute 'predict'](), which makes sense because the self.perturbation_identifier is somehow turning to None after one line. i don't know how that's possible. Any suggestion?