Improving the Transferability of Adversarial Samples with Adversarial Transformations

Wu, Wei; Su, Yuxin; Lyu, Michael R.; King, Irwin

doi:10.1109/cvpr46437.2021.00891

Cited by 59 publications

(28 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second line of defenses is carried out by input transformation to purify the adversarial examples. They preprocess the inputs to cleanse adversarial perturbations without reducing the classification accuracy on clean images (Wu et al 2021). The advanced defenses of this kind include applying random resizing and padding (R&P) (Xie et al 2018), a high-level representation denoiser (HGD) (Liao et al 2018), JPEG compression (JPEG) (Guo et al 2018), feature distillation (FD) (Liu et al 2019b), feature squeezing method: bit reduction (BIT) (Xu, Evans, and Qi 2018) and a neural representation purifier (NRP) (Naseer et al 2020).…”

Section: Defend Against Adversarial Attacksmentioning

confidence: 99%

Improving Adversarial Transferability with Spatial Momentum

Wang¹,

Wei²,

Yan³

2022

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNN) are vulnerable to adversarial examples. Although many adversarial attack methods achieve satisfactory attack success rates under the white-box setting, they usually show poor transferability when attacking other DNN models. Momentum-based attack (MI-FGSM) is one effective method to improve transferability. It integrates the momentum term into the iterative process, which can stabilize the update directions by adding the gradients' temporal correlation for each pixel. We argue that only this temporal momentum is not enough, the gradients from the spatial domain within an image, i.e. gradients from the context pixels centered on the target pixel are also important to the stabilization. For that, in this paper, we propose a novel method named Spatial Momentum Iterative FGSM Attack (SMI-FGSM), which introduces the mechanism of momentum accumulation from temporal domain to spatial domain by considering the context gradient information from different regions within the image. SMI-FGSM is then integrated with MI-FGSM to simultaneously stabilize the gradients' update direction from both the temporal and spatial domain. The final method is called SM 2 I-FGSM. Extensive experiments are conducted on the ImageNet dataset and results show that SM 2 I-FGSM indeed further enhances the transferability. It achieves the best transferability success rate for multiple mainstream undefended and defended models, which outperforms the state-of-the-art methods by a large margin.

show abstract

Section: Defend Against Adversarial Attacksmentioning

confidence: 99%

Improving Adversarial Transferability with Spatial Momentum

Wang¹,

Wei²,

Yan³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…With the improvement of Artificial Intelligence (AI) models, companies tend to deploy AI in real-world applications like autonomous driving and neural machine translation [59]. However, AI software inherits the deficiencies of AI models that they are prone to erroneous behavior given particular inputs [2,4,5,15,21,[76][77][78]. A line of research has been conducted to test AI software systems to address this problem.…”

Section: Related Work 61 Testing Ai Softwarementioning

confidence: 99%

AEON: A Method for Automatic Evaluation of NLP Test Cases

Huang¹,

Zhang²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Due to the labor-intensive nature of manual test oracle construction, various automated testing techniques have been proposed to enhance the reliability of Natural Language Processing (NLP) software. In theory, these techniques mutate an existing test case (e.g., a sentence with its label) and assume the generated one preserves an equivalent or similar semantic meaning and thus, the same label. However, in practice, many of the generated test cases fail to preserve similar semantic meaning and are unnatural (e.g., grammar errors), which leads to a high false alarm rate and unnatural test cases. Our evaluation study finds that 44% of the test cases generated by the state-of-the-art (SOTA) approaches are false alarms. These test cases require extensive manual checking effort, and instead of improving NLP software, they can even degrade NLP software when utilized in model training. To address this problem, we propose AEON for Automatic Evaluation Of NLP test cases. For each generated test case, it outputs scores based on semantic similarity and language naturalness. We employ AEON to evaluate test cases generated by four popular testing techniques on five datasets across three typical NLP tasks. The results show that AEON aligns the best with human judgment. In particular, AEON achieves the best average precision in detecting semantic inconsistent test cases, outperforming the best baseline metric by 10%. In addition, AEON also has the highest average precision of finding unnatural test cases, surpassing the baselines by more than 15%. Moreover, model training with test cases prioritized by AEON leads to models that are more accurate and robust, demonstrating AEON's potential in improving NLP software. CCS CONCEPTS• Software and its engineering → Software testing and debugging.

show abstract

“…Transfer-based adversarial attacks in image classification models often achieve limited success due to the overfitting of local models. However, mechanisms [41,42] have been proposed to circumvent the overfitting issue, promoting the transferability of adversarial samples. While empirical evidence shows that transfer attacks are successful, Demontis et al take it a step further and perform a comprehensive analysis [10] to study underlying reasons for the transferability of attacks.…”

Section: Transfer Attacksmentioning

confidence: 99%

Mitigating Adversarial Attacks by Distributing Different Copies to Different Users

Zhang¹,

Tann²,

Chang³

2021

Preprint

View full text Add to dashboard Cite

Machine learning models are vulnerable to adversarial attacks. In this paper, we consider the scenario where a model is to be distributed to multiple users, among which a malicious user attempts to attack another user. The malicious user probes its copy of the model to search for adversarial samples and then presents the found samples to the victim's model in order to replicate the attack. We point out that by distributing different copies of the model to different users, we can mitigate the attack such that adversarial samples found on one copy would not work on another copy. We first observed that training a model with different randomness indeed mitigates such replication to certain degree. However, there is no guarantee and retraining is computationally expensive. Next, we propose a flexible parameter rewriting method that directly modifies the model's parameters. This method does not require additional training and is able to induce different sets of adversarial samples in different copies in a more controllable manner. Experimentation studies show that our approach can significantly mitigate the attacks while retaining high classification accuracy. From this study, we believe that there are many further directions worth exploring.

show abstract

Improving the Transferability of Adversarial Samples with Adversarial Transformations

Cited by 59 publications

References 22 publications

Improving Adversarial Transferability with Spatial Momentum

Improving Adversarial Transferability with Spatial Momentum

AEON: A Method for Automatic Evaluation of NLP Test Cases

Mitigating Adversarial Attacks by Distributing Different Copies to Different Users

Contact Info

Product

Resources

About