On the human evaluation of universal audio adversarial perturbations

Vadillo, Jon; Santana, Roberto

doi:10.1016/j.cose.2021.102495

Cited by 9 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The present work also joins other projects that have begun to explore similar themes. For example, Vadillo and Santana (2020) use adversarial speech stimuli consisting of regular audio overlaid with specific noise patterns that cause a speech recognition system to misclassify the audio clips. These authors also advocate for careful human subjects testing before describing an adversarial audio attack as “imperceptible,” though their studies primarily focus on the more basic capacity to distinguish clips containing adversarial attacks from normal clips.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Can You Hear Me Now? Sensitive Comparisons of Human and Machine Perception

Lepori¹,

Firestone²

2022

Cognitive Science

View full text Add to dashboard Cite

The rise of machine‐learning systems that process sensory input has brought with it a rise in comparisons between human and machine perception. But such comparisons face a challenge: Whereas machine perception of some stimulus can often be probed through direct and explicit measures, much of human perceptual knowledge is latent, incomplete, or unavailable for explicit report. Here, we explore how this asymmetry can cause such comparisons to misestimate the overlap in human and machine perception. As a case study, we consider human perception of adversarial speech — synthetic audio commands that are recognized as valid messages by automated speech‐recognition systems but that human listeners reportedly hear as meaningless noise. In five experiments, we adapt task designs from the human psychophysics literature to show that even when subjects cannot freely transcribe such speech commands (the previous benchmark for human understanding), they can sometimes demonstrate other forms of understanding, including discriminating adversarial speech from closely matched nonspeech (Experiments 1 and 2), finishing common phrases begun in adversarial speech (Experiments 3 and 4), and solving simple math problems posed in adversarial speech (Experiment 5) — even for stimuli previously described as unintelligible to human listeners. We recommend the adoption of such “sensitive tests” when comparing human and machine perception, and we discuss the broader consequences of such approaches for assessing the overlap between systems.

show abstract

Section: Discussionmentioning

confidence: 99%

“…99We note that Vadillo and Santana (2020) also demonstrate that subjects found some adversarial stimuli to be less natural than nonadversarial stimuli. This difference is more in line with the present work, as it demonstrates subjects' perception of specific differences between clips that may signal that the stimuli contain an adversarial attack.…”

mentioning

confidence: 99%

Can You Hear Me Now? Sensitive Comparisons of Human and Machine Perception

Lepori¹,

Firestone²

2022

Cognitive Science

View full text Add to dashboard Cite

show abstract

“…However, it is unsuitable for the speech-impaired and is susceptible to environmental noise interference. In contrast, lip-reading recognition (LRR) technology, which enables silent speech recognition, has gained significant research interest in various fields, including speech-impaired communication and human–machine interaction (HMI). , …”

Section: Introductionmentioning

confidence: 99%

“…In contrast, lip-reading recognition (LRR) technology, which enables silent speech recognition, 1 has gained significant research interest in various fields, including speech-impaired communication and human−machine interaction (HMI). 2,3 Unlike traditional speech recognition methods that rely on computer vision or millimeter-wave radar technology, 4,5 LRR is less affected by environmental noise but faces challenges in large-scale production due to complex, costly instrumentation systems, and intricate processing procedures. Additionally, computer vision-based LRR is easily influenced by ambient light, resulting in reduced recognition rates under insufficient lighting conditions.…”

Section: Introductionmentioning

confidence: 99%

Laser-Induced Graphene Strain Sensor for Conformable Lip-Reading Recognition and Human–Machine Interaction

Cheng

Fang

Wei

et al. 2023

ACS Appl. Nano Mater.

View full text Add to dashboard Cite

Lip-reading recognition (LRR) has gained significant attention due to its potential applications in various scenarios, such as communication for the speech-impaired, conversations in noisy or dark environments, and human–machine interactions. However, existing LRR technologies based on computer vision suffer from drawbacks such as the high cost of electronic camera equipment and the negative impact of ambient lighting on recognition accuracy. Herein, a graphene-based flexible strain sensor is developed through a facile, high-efficiency, and low-cost laser-induced carbonization technique, which involves the ablation of polyimide (PI) films using ultraviolet lasers. The sensor’s patterned stripes and porous structure endow it with sensitivity to deformations caused by bending and pressing. The well-designed flexible strain sensor can tightly attach on facial skin and record high-quality strain signals of various lip muscle movements. When compared to a preconstructed lip-reading database using a fixed algorithm, the collected lip-reading signals exhibit a recognition rate exceeding 90%, enabling seamless human–machine interaction and precise control over manipulators. Consequently, the LRR approach based on the flexible strain sensor demonstrates immense potential as a promising platform for speech-impaired communication and human–machine interactions in variable environments.

show abstract