softsys4ai / athena Goto Github PK

This will be a baseline attack, also we compare with a model trained on random noise.

Hi I'm trying to reproduce your experiment but while I was trying to evaluate FGSM attacks under white box settings, I found that the normalized l2-dissimilarity for a FGSM at eps=0.1 is only about 0.007. As in your code, the upper bound for an white-box attack is determined by some pre-generated adversarial examples. I'm wondering how you guys finish the experiments as in Fig. 7 in your paper?

So far, we have empirical evidence that shows there is a significant difference between BS and AE in terms of inference time on the clean model and many transform models. Check the column "inference Probability" in
Inference_Time-T_Test.xlsx for the t-test result of inference time of BS and AE on every model.

Question: what is the root cause of this difference?

Investigation idea:

1. Which part of NN layers (lower layers or upper layers) has more impact on this difference?
   measure the time cost for each layer for both BS and AE

2. Do the inferences of BS and AE result in a different number of neurons to be activated

we need to prepare a python notebook containing our end to end approach with some specific examples

white-box

[Error Message]

Traceback (most recent call last):
File "detection_as_defense.py", line 67, in
transformationList)
File "/home/kevinsjh_gmail_com/adversarial_transformers/util.py", line 1087, in predictionForTest
tranSamples = transform_images(curSamples, transformType)
File "/home/kevinsjh_gmail_com/adversarial_transformers/transformation.py", line 748, in transform_images
elif (transformation_type in TRANSFORMATION.NOISES):
File "/home/kevinsjh_gmail_com/adversarial_transformers/transformation.py", line 671, in add_noise
img_noised = skimage.util.random_noise(img, mode=noise_mode)
File "/home/kevinsjh_gmail_com/anaconda3/lib/python3.7/site-packages/skimage/util/noise.py", line 155, in random_noise
out = np.random.poisson(image * vals) / float(vals)
File "mtrand.pyx", line 4005, in mtrand.RandomState.poisson
ValueError: lam value too large.

After investigating the accuracy of all models (clean model + 32 transform models) under the attack of FGSM250 AEs, it is observed that the accuracy of a few transform models become lower than the clean model's accuracy, which are higher than that of the clean model previously. Check the table below.

We should use t-sne to look into some properties of models trained on transformations and provide some intuitions why they work.

Use logging to manage the information, rather than using print function.

1. Use ensemble models trained from fgsm AEs and have them tested on bim AEs.

2. Use ensemble models trained from bim AEs and have them tested on fgsm AEs.

1. Some causes crash while some generated black images.
   list:
   new filter transformations,
   denoising transformations,
   geo transformations,
   seg transformations.

2. Rename some of the existing transformation types to organize all transformations better.

unit tests

1. filter.sobel
2. all augment transformations.

We'd like to provide one-pixel attack in our tool kit.

provide CW (0, 2, and inf norms) attacks

We'd like to provide a simple blackbox attack

use FLAG to manage

1. project configurations
2. global variables
3. other related values/settings.

Presume that attackers have access to all weak defenses but are not aware of the ensemble strategies.

pgd attack

We should investigate how adv attacks change the class activation mapping (CAM) and how our transformations change the map. It may reveal some understanding of why our approach works.

support deepfool (L_inf norm)

Approach 1: add randomness (random noise) to AEs and then use them to train ensemble models

Approach 2: use the strongest type of AEs to build ensemble models for defense

U of SC computing resource center supports Sylab.io, With that, we should be able to create a docker container, which gives us root privilege, and deploy it to the computing resource center to access GPU nodes.

Try it out after conferences' deadlines

The scripts, which work well on local machines, do not work on rci nodes, some exceptions thrown by tensorflow. Highly suspect this is caused by the gap between versions. I am using tensorflow 1.13 on local machines, while tensorflow 1.12 is on rci nodes.

Need to take a look at this issue and fix the bugs on rci.

support MIM attack

Hi, is it possible to implement reset function for other transformations?

With some figures and appropriate plots.

The idea is to differentiate BS and AE based on their output from distinct transform models.

In Detecing adversarial samples from artifacts.pdf, it is shown that different models make different mistakes when presented to same AEs. And 2018-arXiv-PictureAE-Picture_AE_detection_bimodel.pdf proposes Bi-model approach that concatenates the output of an image from two distinct models as its feature representation and then feeds it to a binary classifier for classification. The approach is claimed to reach >90% detection accuracy on mnist and cifar10.

we can concatenate/stack up the output of transform models for an input image and use it as a representation of the image and feed into a binary classifier. This might could have a higher detection accuracy and generalize better across different type of attacks.

Investigation:

1. Identify patterns of the prediction output of BS and AE
   for BS and each type of AE, plot the boxplot for the average, min and max accuracy of all transform models

2. Detection approach 1: majority voting
   the prediction output of AEs is much more diverse than that of BS. That is, the number of transform models agrees with each other on AE will be much smaller than that on BS. If this number is below some threshold, say 75% X total_number_of_models, the input image will be marked as an AE. Otherwise, it is considered as a benign sample.

3. Detection approach 2: distance matrix
   【empirical evidence to collection】: distance of prediction outputs of a benign sample from two distinct transform models is close to 0, while the distance of prediction outputs of an AE from two distinct transform models should be much larger than 0.
   Try with different distance metrics: L2, entropy, KL divergence, cosine, correlation

   【distance matrix】: for an image, create a distance matrix by computing the distances of its prediction outputs between each pair of transform models. Investigate any possible property or difference between AE and BS