Adversarial Machine Learning: A Systematic Survey of Backdoor Attack, Weight Attack and Adversarial Example

Abstract

Adversarial machine learning (AML) studies the adversarial phenomenon of machine learning, which may make inconsistent or unexpected predictions with humans. Some paradigms have been recently developed to explore this adversarial phenomenon occurring at different stages of a machine learning system, such as training-time adversarial attack (i.e., backdoor attack), deployment-time adversarial attack (i.e., weight attack), and inference-time adversarial attack (i.e., adversarial example). However, although these paradigms share a common goal, their developments are almost independent, and there is still no big picture of AML. In this work, we aim to provide a unified perspective to the AML community to systematically review the overall progress of this field. We firstly provide a general definition about AML, and then propose a unified mathematical framework to covering existing attack paradigms. According to the proposed unified framework, we can not only clearly figure out the connections and differences among these paradigms, but also systematically categorize and review existing works in each paradigm.

Invisible Backdoor Attack against 3D Point Cloud Classifier in Graph Spectral Domain

“…Backdoor attack is firstly proposed in 2D image domain (Gu, Dolan-Gavitt, and Garg 2017). Then, a lot of attacks was developed (Zhao et al 2022;Wu et al 2023;Feng et al 2022;Yuan et al 2023;Doan et al 2023). However, due to data format restriction, attacks for 2D image cannot directly apply to 3D point cloud.…”

Unleashing the Power of Indirect Attacks against Trust Prediction via Preferential Path

“…Consequently, the proposed vicinage-attack can be seen as a vulnerability test for trust systems. Reference [40] outlines three basic conditions for adversarial attacks: stealthiness, consistency, and inconsistency. The vulnerability of a trust system is also reflected in these three aspects.…”

Benchmarking Adversarial Patch Against Aerial Detection

“…For instance, a model could yield significantly different predictions on two visually similar images, with one being perturbed by malicious and imperceptible noises [15], [16], whereas a human's prediction would remain unaffected by such noises. We refer to this phenomenon as the adversarial phenomenon or adversarial attack, signifying the inherent adversarial relationship between DL models and human perception [28].…”

“…To comprehensively assess the robustness of these models, we conducted rigorous experiments involving a diverse set of classifiers and detectors, representing a wide range of mainstream methods. Through this extensive evaluation, we have uncovered insightful and intriguing findings that illuminate the relationship between the crafting of [32] IEEE Access ✓ ✕ ✕ ✕ ✕ ✕ 2018 [34] Computer Science Review ✓ ✕ ✕ ✕ ✕ ✕ 2018 [31] arXiv ✓ ✕ ✕ ✕ ✕ ✕ 2019 [33] Applied Science ✓ ✕ ✕ ✕ ✕ ✕ 2020 [42] ACM Computing Surveys ✓ ✕ ✕ ✕ ✕ ✕ 2021 [35] IEEE Access ✓ ✕ ✕ ✕ ✕ ✕ 2021 [41] ACM Computing Surveys ✓ ✕ ✕ ✕ ✕ ✕ 2021 [40] TII ✓ ✕ ✕ ✕ ✕ ✕ 2022 [47] arXiv ✓ * ✕ ✕ ✕ ✕ 2022 [48] INJOIT ✓ ✕ ✕ ✕ ✕ ✕ 2022 [49] Artificial Intelligence Review ✓ ✕ ✓ ✕ ✓ ✕ 2022 [39] TPAMI ✓ ✕ ✕ ✕ ✕ ✕ 2022 [38] TII ✕ ✕ ✕ ✕ ✕ ✕ 2022 [49] arXiv ✓ * ✕ ✕ ✕ ✕ 2022 [25] arXiv ✓ ✕ ✕ ✕ ✕ ✕ 2022 [44] arXiv ✓ * ✕ ✕ ✕ ✕ 2022 [45] arXiv * ✓ ✕ ✕ ✕ ✕ 2022 [37] Neurocomputing ✓ ✕ ✕ ✕ ✕ ✕ 2023 [50] ACM Computing Surveys * ✕ ✕ ✕ ✕ ✕ 2023 [28] arXiv ✓ ✕ ✕ ✕ ✕ ✕ 2023 [46] ICAI * ✓ ✕ ✕ ✕ ✕ Benchmarks 2020 [29] CVPR ✕ ✕ ✓ ✕ ✓ ✕ 2021 [27] arXiv ✕ ✕ ✓ ✕ ✓ ✓ 2022 [51] IJCAI ✕ ✕ ✕ ✓ ✓ ✕ 2022 [26] NIPS ✕ ✕ ✓ ✕ ✓ ✕ 2022 [52] arXiv ✕ ✕ ✕ ✓ ✓ ✕ 2022 [36] Pattern Recognition ✕ ✕ ✓ ✕ ✓ ✕ 2023 [30] arXiv ✕ ✕ ✓ ✕ ✓ ✓ 2023 [53] Pattern Recognition ✕ ✕ ✓ ✕ ✓ ✕ 2023 [54] CVPR…”