On the human evaluation of audio adversarial examples

Abstract

Human-machine interaction is increasingly dependent on speech communication. Machine Learning models are usually applied to interpret human speech commands. However, these models can be fooled by adversarial examples, which are inputs intentionally perturbed to produce a wrong prediction without being noticed. While much research has been focused on developing new techniques to generate adversarial perturbations, less attention has been given to aspects that determine whether and how the perturbations are noticed by humans. This question is relevant since high fooling rates of proposed adversarial perturbation strategies are only valuable if the perturbations are not detectable. In this paper we investigate to which extent the distortion metrics proposed in the literature for audio adversarial examples, and which are commonly applied to evaluate the effectiveness of methods for generating these attacks, are a reliable measure of the human perception of the perturbations. Using an analytical framework, and an experiment in which 18 subjects evaluate audio adversarial examples, we demonstrate that the metrics employed by convention are not a reliable measure of the perceptual similarity of adversarial examples in the audio domain.

Can You Hear Me Now? Sensitive Comparisons of Human and Machine Perception

“…(Means and SDs differed across the experiments, with Experiment 2 yielding worse overall performance but less variance and more included subjects.) 9 We note that Vadillo and Santana (2020) also demonstrate that subjects found some adversarial stimuli to be less natural than nonadversarial stimuli. This difference is more in line with the present work, as it demonstrates subjects' perception of specific differences between clips that may signal that the stimuli contain an adversarial attack.…”

“…The present work also joins other projects that have begun to explore similar themes. For example, Vadillo and Santana (2020) use adversarial speech stimuli consisting of regular audio overlaid with specific noise patterns that cause a speech recognition system to misclassify the audio clips. These authors also advocate for careful human subjects testing before describing an adversarial audio attack as "imperceptible," though their studies primarily focus on the more basic capacity to distinguish clips containing adversarial attacks from normal clips.…”

Personal Voice Assistant Security and Privacy—A Survey

“…Research in this area could help building more effective adversarial example studies. We notice that there is a recent work [93] developing this area.…”