jinhaoduan / secmi Goto Github PK
View Code? Open in Web Editor NEW[ICML 2023] Are Diffusion Models Vulnerable to Membership Inference Attacks?
License: Do What The F*ck You Want To Public License
[ICML 2023] Are Diffusion Models Vulnerable to Membership Inference Attacks?
License: Do What The F*ck You Want To Public License
Hi, I am doing some research on the privacy of diffusion models. Could I get the code you use to test GAN-Leaks?
Though you've explained it in issue6, I'd still like to double-check this detail cause I'm confused about the experiement result.
Referring to the official code for stable diffusion fine-tuning and the .sh example, there are two parameters for data transform during fine-tuning, namely args.center_crop and args.random_flip, representing centerCrop / randomCrop, no-Flip / randomFlip respectively.
If using the default parameter settings, it's actually a combination of randomCrop and no-Flip. Here's the code:
# Preprocessing the datasets.
train_transforms = transforms.Compose(
[
transforms.Resize(args.resolution, interpolation=transforms.InterpolationMode.BILINEAR),
transforms.CenterCrop(args.resolution) if args.center_crop else transforms.RandomCrop(args.resolution),
transforms.RandomHorizontalFlip() if args.random_flip else transforms.Lambda(lambda x: x),
transforms.ToTensor(),
transforms.Normalize([0.5], [0.5]),
]
)
I'd like to know which parameter combination you used?
I conducted experiments on various data-augmentation combination based on your methods (COCO dataset, 2500/2500 split, 150,000 steps, as described in your paper).
The experimental results align with yours only for randomCrop+ no-Flip (ASR/AUC: 0.8334/0.9105). And centerCrop +no-Flip yields higher results (ASR≈0.90). On the other hand, randomCrop+randomFlip results in much lower performance (ASR≈0.75).
So, did you use the combination of randomCrop and no-Flip?
Thank you for your time, and I look forward to your response!
Thank you for your work and for releasing the model weights
However, I obtained different results using your method on the official CIFAR-10 diffusion model ddpm-cifar10-google with the cifar10 official train/test split.
Could you please release the training/testing dataset corresponding to your pretrained ddpm model?
Thank you for your amazing work!
I'd like to inquire if there are evaluation codes about conditional diffusion model in this repository (Sec. 5.5 and Sec. 5.6 in paper) ?
Thank you for your inspiring work!
But I have one more question that arises from the closed issue:
You can assume we are in a shadow environment: training a shadow model with a known member/nonmember splitting and determining a threshold/NN according to this known splitting. After we get the threshold/NN, we could apply them to any real victim models (which we don't know the membership splitting).
We have some results to show that the threshold obtained from the shadow environment can be transferred to the real victim models:
Method Attack Target AUC ASR
SecMI_stat Shadow Model 0.881 0.811
SecMI_stat Victim Model - 0.804
SecMI_NNs Shadow Model 0.951 0.888
SecMI_NNs Victim Model - 0.893
Could you please describe how to construct the training data for the "shadow models"? When training a shadow model, do you assume that the adversary already knows the distribution of the dataset? (Or can obtain some of the training data?)
Hi,
Thanks for your great work! I have a question about the evaluation process. In my understanding, current implementations directly use the member/non-member labels to get the threshold for
Line 241 in e216395
Hi, thank you for your paper and for sharing the code. May I ask some questions?
Line 36 in 2560350
epsilon
, initially t_c
is 0
, but during DDPM inference, time step t_c
should be equal or great than 1
. So can we set t_c
to 0 here?
Thanks.
Thank you for your great work and open-source code, which inspires me a lot.
During replication, there was a slight disparity between my results and yours (with ASR, AUC even higher on the Pokémon dataset than yours). So I want to know which differing settings led to my higher results.
My setting:
Pokmon train-test split: 416, 417.
Training steps: 15000, Batch size: 1, Gradient_accumulation_steps: 4, LR: 1e-5.
Without Crop and Flip. (Did you use crop and flip during training?)
My result
ASR 0.90, AUC 0.9391 with Prompt (higher than yours: 0.821, 0.891)
Trying to keep the settings consistent with the paper, but I still obtained different results. Looking forward to your response!
Dear Authors,
Thanks for your excellent work. I've noted the statistics for CIFAR10 in ./stats/cifar10.train.npz. Would it be feasible for you also to release the statistics for CIFAR100 and Tiny-ImageNet?
Thank you for your time and consideration.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.