Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability [NeurIPS'2023]
Haotian Xue 1, Alexandre Araujo 2, Bin Hu 3, and Yongxin Chen 1
1 GaTech, 2 NYU, 3 UIUC
Diff-PGD utilizes strong prior knowledge of Diffusion Model to generate adversarial samples with higher steathiness and controllability. Diff-PGD has the following edges:
- Generate adversarial samples with higher steathiness and controllability
- Can be easily applied to different scenarios (e.g. global, region, style-based, physical world)
- Higher transferability and anti-purification power
- Introduction
- News
- Envs
- Run Global Attack
- Run Regional Attack
- Run Style-based Attack
- Run Physical World Attack
- Some other useful code descriptions
⭐ [2023-09-21] Diff-PGD is accepted by NeurIPS 2023!
⭐ [2023-09-21] Accelerated version (v2) can be now used
⭐ [2023-06-03] We release the code!
⭐ [2023-05-25] Our paper is released: https://arxiv.org/abs/2305.16494
- Creat conda env using
env.yml:
conda env create -f env.yml
- Activate conda env:
source activate diff-pgd
-
Download [DM checkpoint] and put them into
ckpt/ -
Prepare imagenet dataset, and in
code/dataset.py:17, replace it with:
os.environ['IMAGENET_LOC_ENV'] = YOUR_IMAGENET_PATH
Here we provide an example to run gobal attack for imagenet val set:
python code/attack_global.py
or you can import Attack_Global from code/attack_global.py to build your customized settings:
Attack_Global(classifier, device, respace, t, eps, iter, name, alpha, version, skip)
[classifier]: target classifier to attack
[device]: device, gpu id
[respace] diffusion model accelerator, e.g. ddim10
[t] reverse step in SDEdit
[eps] l_inf budget of PGD
[iter] iterations number of PGD
[name] name of the exp
[alpha] step size of PGD
[version] base version v1, v2 is a faster version to be released
[skip] skip interval of dataset
Here we provide an example to run gobal attack for imagenet val set:
python code/attack_region.py
similarly, we have the settings as:
Attack_Region(classifier, device, respace, t, eps, iter, name, skip, ratio):
[classifier]: target classifier to attack
[device]: device, gpu id
[respace] diffusion model accelerator, e.g. ddim10
[t] reverse step in SDEdit
[eps] l_inf budget of PGD
[iter] iterations number of PGD
[name] name of the exp
[alpha] step size of PGD
[skip] skip interval of dataset
[ratio] ratio for masked square region, size ratio*H, ratio*W
We provide some examples of style-based adversarial attack (mostly from AdvCAM official repo) in data/advcam, include target image, style reference image and segmentation masks.
In the code example code/attack_style.py, we define some scenes like ['tra4', 'tra1', 'tra5', 'tra6', 'leaf', 'car', 'lamp', 'car'], we also encourage users to define their own stlye-based attack settings.
python code/attack_style.py
the core function has settings like:
Attack_Region_Style(exp_name, classifier, device, respace, t, eps, iter, name):
[exp_name]: defined name of example, e.g. tra5, tra4, car
[classifier]: target classifier to attack
[device]: device, gpu id
[respace] diffusion model accelerator, e.g. ddim10
[t] reverse step in SDEdit
[eps] l_inf budget of PGD
[iter] iterations number of PGD
[name] name of the exp
We provide some physical world attack data in data/our_dataset, including background and pathes in the paper:
- backbag attack background:
data/our_dataset/bag - computer mouse attack background:
dataset/our_dataset/computer_mouse - patches and masks:
data/our_dataset/patches
to run example code:
python code/attack_physics.py
core function and settings for computer mouse as an example:
Attack_Physics(mode='diff-pgd', bkg_name='computer_mouse', patch_name='apple', classifier='resnet50', device=0, respace='ddim10', t=2, target=None, c_w=1, s_m=0.01, iter=4000, name='attack_physics')
code/guided_diffusion/: official implementation of guided_diffusioncode/dataset.py: load imagenet datasetcode/load_dm.py: load diffusion model used in Diff-PGDcode/attack_tools.py: some adversarial attack toolscode/stat.py: run statistics on success rate, transferability and anti-purification power [to be released]
@article{xue2023diffusion,
title={Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability},
author={Xue, Haotian and Araujo, Alexandre and Hu, Bin and Chen, Yongxin},
journal={arXiv preprint arXiv:2305.16494},
year={2023}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent