Practical Hidden Voice Attacks against Speech and Speaker Recognition Systems

Abstract

Voice Processing Systems (VPSes), now widely deployed, have been made significantly more accurate through the application of recent advances in machine learning. However, adversarial machine learning has similarly advanced and has been used to demonstrate that VPSes are vulnerable to the injection of hidden commands -audio obscured by noise that is correctly recognized by a VPS but not by human beings. Such attacks, though, are often highly dependent on white-box knowledge of a specific machine learning model and limited to specific microphones and speakers, making their use across different acoustic hardware platforms (and thus their practicality) limited. In this paper, we break these dependencies and make hidden command attacks more practical through model-agnostic (blackbox) attacks, which exploit knowledge of the signal processing algorithms commonly used by VPSes to generate the data fed into machine learning systems. Specifically, we exploit the fact that multiple source audio samples have similar feature vectors when transformed by acoustic feature extraction algorithms (e.g., FFTs). We develop four classes of perturbations that create unintelligible audio and test them against 12 machine learning models, including 7 proprietary models (e.g., Google Speech API, Bing Speech API, IBM Speech API, Azure Speaker API, etc), and demonstrate successful attacks against all targets. Moreover, we successfully use our maliciously generated audio samples in multiple hardware configurations, demonstrating effectiveness across both models and real systems. In so doing, we demonstrate that domain-specific knowledge of audio signal processing represents a practical means of generating successful hidden voice command attacks.

SurfingAttack: Interactive Hidden Attack on Voice Assistants Using Ultrasonic Guided Waves

“…4(b). We use a chirp signal from 50 Hz to 2 Out-of-plane displacement is defined as the displacement along the x 3 direction. 3 In-plane displacement is defined as the displacement along the x 1 direction.…”

“…With the rapidly growing popularity and functionality of voice-driven IoT devices, voice-based attacks have become a non-negligible security risk. Gong et al investigate and classify voice-based attacks [20] into four major categories: basic voice replay attacks [12], [29], [36], operating system level attacks [3], [15], [26], [53], machine learning level attacks [2], [9], [10], [13], [19], [43], [48], [51], and hardware level attacks [28], [52]. A machine learning level attack uses adversarial audio commands to attack automatic speech recognition (ASR) systems.…”

Manipulating Users’ Trust on Amazon Echo: Compromising Smart Home from Outside

“…For instance, Wifi typically works great around 10 meters, Bluetooth within several meters, and NFC around 10 centimeters. It is clear that the speakers only supporting NFC are not ideal for remote hacking, since it requires 7 EAI Endorsed Transactions on Security and Safety 08 2019 -05 2020 | Volume 6 | Issue 22 | e3 the attackers be in the home close enough to the speaker. Regarding speakers supporting Bluetooth or Wifi, once attackers could retain a short distance with them outside the home, the speakers will be visible to attackers' audio devices (either Bluetooth or Wifi capable).…”

“…Vaidya et al [52] and Carlini et al [16] observed that attackers could issue hidden voice commands which were unrecognizable to human listeners but can be interpreted as desired commands by CMU Sphinx speech system, also in their blackbox attack, the voice commands can be understood by Google Speech API. Similarly, Hadi et al [7] use four methods to generate the noisy audios to practically attack several speech recognition models. Yuan et al [56] successfully embedded voice commands into regular songs stealthily, which can compromise Kaldi, one popular open-sourced speech recognition system.…”

“…Based on the other observations, researchers recently designed novel attacks against speech recognition systems like Alexa, that is, either uninterpretable [7,16,56] or inaudible [57] voice commands to human, but recognizable by speech recognition systems. Though seminal, these works typically require a speaker or an ultrasound transducer be close enough to the target, e.g., Echo, to broadcast the obfuscated voice.…”