Adversarial Examples: Opportunities and Challenges

Abstract

Deep neural networks (DNNs) have shown huge superiority over humans in image recognition, speech processing, autonomous vehicles and medical diagnosis. However, recent studies indicate that DNNs are vulnerable to adversarial examples (AEs) which are designed by attackers to fool deep learning models. Different from real examples, AEs can mislead the model to predict incorrect outputs while hardly be distinguished by human eyes, therefore threaten security-critical deep-learning applications. In recent years, the generation and defense of AEs have become a research hotspot in the field of artificial intelligence (AI) security. This article reviews the latest research progress of AEs. First, we introduce the concept, cause, characteristics and evaluation metrics of AEs, then give a survey on the state-of-the-art AE generation methods with the discussion of advantages and disadvantages. After that, we review the existing defenses and discuss their limitations. Finally, future research opportunities and challenges on AEs are prospected.

Robustness to adversarial examples can be improved with overfitting

“…The two major lines of research around adversarial examples have been: (1) generating AEs and (2) defending against AEs. This paper will not cover either, and the reader is referred to recent surveys [1,6,40]. In parallel to those two lines, however, a significant body of work has been carried out to delve into the root causes of AEs and their implications.…”

Review of Artificial Intelligence Adversarial Attack and Defense Technologies

“…The probability of class "1" before and after perturbation. Before perturbating, the score of class "1" is 5%; however, the score is increased to 88% by adding or subtracting 0.5 from each dimension in a specific direction of the original sample (Image Credit: Zhang et al [59]).…”

“…Adversarial samples have three basic characteristics [59], i.e., transferability, regularization effect, and adversarial instability.…”

Towards Resistant Audio Adversarial Examples

“…With the rise of neural networks, adversarial examples and their mitigation [7,16,39] have become subject to competitive research, especially in the image domain, see Section 2.1. Analog to images, one can generate adversarial examples for audio data, see Section 2.2.…”

“…The first adversarial examples to non-linear algorithms were generated by Biggio et al [4] and Szegedy et al [31] in 2013. Up until now, various attacks on plenty of datasets focusing on images have been presented (for an overview, see, e.g., [39]). Later, adversarial audio started to get analysed as well [10,12,17,25,26,33,35].…”