MLHospital is a repo to evaluate inference attacks and the corresponding defenses against machine learning models.

Currently we support membership inference attacks and attribute inference attacks.

git clone https://github.com/TrustAIResearch/MLHospital.git;
cd MLHospital;
conda env create -f environment.yml;
conda activate ml-hospital;
python setup.py install;

cd MLHospital/mlh/examples;
python train_target_models.py --mode target;
python train_target_models.py --mode shadow;

Note that you can also specify the --training_type with different defense mechanisms, e.g., Normal, LabelSmoothing, AdvReg, DP, MixupMMD, and PATE.

python mia_example.py 

Note that you can also specify the --attack_type with different attacks, e.g., black-box, black-box-sorted, black-box-top3, metric-based, and label-only.

cd MLHospital/mlh/examples;
python3 aia_example.py --task aia --dataset CelebA --defense AdvTrain --alpha 1.0;

The aia_example.py first trains target models (with or without defense), then trains and evaluates the attack model.

In this example, CelebA dataset is used, and the defense method and alpha (the hyperparameter to balance utility and privacy) are set to be AdvTrain and 1.0, respectively. Note that you can also specify the --defense with different defense mechanisms, e.g., Normal, AdvTrain, Olympus, and AttriGuard.

The tool is designed and developed by Xinlei He (CISPA), Zheng Li (CISPA), Yukun Jiang (CISPA), Yun Shen (NetApp), and Yang Zhang (CISPA).