Wild patterns: Ten years after the rise of adversarial machine learning

Abstract

Learning-based pattern classifiers, including deep networks, have shown impressive performance in several application domains, ranging from computer vision to cybersecurity. However, it has also been shown that adversarial input perturbations carefully crafted either at training or at test time can easily subvert their predictions. The vulnerability of machine learning to such wild patterns (also referred to as adversarial examples), along with the design of suitable countermeasures, have been investigated in the research field of adversarial machine learning. In this work, we provide a thorough overview of the evolution of this research area over the last ten years and beyond, starting from pioneering, earlier work on the security of non-deep learning algorithms up to more recent work aimed to understand the security properties of deep learning algorithms, in the context of computer vision and cybersecurity tasks. We report interesting connections between these apparently-different lines of work, highlighting common misconceptions related to the security evaluation of machine-learning algorithms. We review the main threat models and attacks defined to this end, and discuss the main limitations of current work, along with the corresponding future challenges towards the design of more secure learning algorithms. (Fabio Roli) 1 The term malware (short for malicious software) is normally used to refer to harmful computer programs in general, including computer viruses, worms, ransomware, spyware, etc.2 More than 150 papers on this subject were published on ArXiv only in the last two years.

Hardening quantum machine learning against adversaries

“…As machine learning becomes more ubiquitous in applications so too do attacks on the learning algorithms that they are based on [5][6][7][8][9][10][11][12][14][15][16][17]. The key assumption usually made in machine learning is that the training data is independent of the model and the training process.…”

“…While a laundry list of attacks are known against machine learning systems, the defences that have been developed thus far are somewhat limited [9,43]. A commonly used tactic is to replace classifiers with robust versions of the same classifier.…”

“…The study of subverting machine learning systems by motivated attackers is called adversarial machine learning and according to [9] dates to 2004. Adversaries can reduce the confidence of prediction, replicate the underlying model [8], recover the training data [10], backdoor models to fail on specific examples [11] and even reprogram the model to a completely different task [12].…”

Polarizing Front Ends for Robust Cnns

“…Given the immense impact of deep learning on a diversity of fields, its vulnerability to tiny adversarial perturbations [1,2] is of great concern. For image datasets, for example, such perturbations are almost imperceptible for humans, but they can render state-of-the-art models useless, causing misclassification with high confidence.…”

“…Attacks: State of the art ∞ bounded attacks (used in our evaluations) are all based on gradient ascent on the cost function in (1). The Fast Gradient Sign Method (FGSM) [3], computes the perturbation by e = · sign(∇ x L(θ, x, y))…”