Robust Principles: Architectural Design Principles for Adversarially Robust CNNs. ShengYun Peng, Weilin Xu, Cory Cornelius, Matthew Hull, Kevin Li, Rahul Duggal, Mansi Phute, Jason Martin, Duen Horng Chau. British Machine Vision Conference (BMVC), 2023.

📺 Video Presentation      📖 Research Paper      🚀Project Page      🪧 Poster

We aim to unify existing works' diverging opinions on how architectural components affect the adversarial robustness of CNNs. To accomplish our goal, we synthesize a suite of three generalizable robust architectural design principles: (a) optimal range for depth and width configurations, (b) preferring convolutional over patchify stem stage, and (c) robust residual block design through adopting squeeze and excitation blocks and non-parametric smooth activation functions. Through extensive experiments across a wide spectrum of dataset scales, adversarial training methods, model parameters, and network design spaces, our principles consistently and markedly improve AutoAttack accuracy: 1-3 percentage points (pp) on CIFAR-10 and CIFAR-100, and 4-9 pp on ImageNet.

Aug. 2023 - Paper accepted by BMVC'23

Sep. 2023 - 🎉 We are the top on RobustBench CIFAR-10 $\ell_\infty = 8/255$ leaderboard

1. Prepare ImageNet following installation steps 3&4. Skip step 4 if you don't plan to run Fast adversarial training (AT).
2. Set up python environment:

make .venv_done

3. (Optional) Register Weights & Biases account if you want to visualize training curves.
4. Update "BASE" to ImageNet root directory and "WANDB_ACCOUNT" to your account name and validate by:

make check_dir

make experiments/Torch_ResNet50/.done_test_pgd

To test other off-the-shelf models in torchvision, add the model name in MODEL.mk and create a new make target in Makefile.

make experiments/RaResNet50/.done_test_pgd

# Training
make experiments/RaResNet50/.done_train

# Evaluation on PGD
make experiments/RaResNet50/.done_test_pgd

# Evaluation on AutoAttack
make experiments/RaResNet50/.done_test_aa

# Pretrained models evaluated on AutoAttack
make experiments/RaResNet50/.done_test_pretrained

@article{peng2023robust,
  title={Robust Principles: Architectural Design Principles for Adversarially Robust CNNs},
  author={Peng, ShengYun and Xu, Weilin and Cornelius, Cory and Hull, Matthew and Li, Kevin and Duggal, Rahul and Phute, Mansi and Martin, Jason and Chau, Duen Horng},
  journal={arXiv preprint arXiv:2308.16258},
  year={2023}
}

@misc{peng2023robarch,
      title={RobArch: Designing Robust Architectures against Adversarial Attacks}, 
      author={ShengYun Peng and Weilin Xu and Cory Cornelius and Kevin Li and Rahul Duggal and Duen Horng Chau and Jason Martin},
      year={2023},
      eprint={2301.03110},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

If you have any questions, feel free to open an issue or contact Anthony Peng (CS PhD @Georgia Tech).