How the Softmax Output is Misleading for Evaluating the Strength of Adversarial Examples

Abstract

Even before deep learning architectures became the de facto models for complex computer vision tasks, the softmax function was, given its elegant properties, already used to analyze the predictions of feedforward neural networks. Nowadays, the output of the softmax function is also commonly used to assess the strength of adversarial examples: malicious data points designed to fail machine learning models during the testing phase. However, in this paper, we show that it is possible to generate adversarial examples that take advantage of some properties of the softmax function, leading to undesired outcomes when interpreting the strength of the adversarial examples at hand. Specifically, we argue that the output of the softmax function is a poor indicator when the strength of an adversarial example is analyzed and that this indicator can be easily tricked by already existing methods for adversarial example generation.

Not All Adversarial Examples Require a Complex Defense: Identifying Over-optimized Adversarial Examples with IQR-based Logit Thresholding

“…We refer the reader to our previous work which details the masking effect of the softmax function in the case of both single-target and multi-target adversarial attacks [21].…”

“…However, when investigating adversarial examples, the softmax function, when used in settings with limited decimal precision, has a major drawback for correctly interpreting the predictions of a neural network: it lacks sensitivity to positive changes in the magnitude of the largest logit. We detailed this discovery and its impact on adversarial examples in a previous study [21].…”