Protecting Voice Controlled Systems Using Sound Source Identification Based on Acoustic Cues

Abstract

Over the last few years, a rapidly increasing number of Internet-of-Things (IoT) systems that adopt voice as the primary user input have emerged. These systems have been shown to be vulnerable to various types of voice spoofing attacks. Existing defense techniques can usually only protect from a specific type of attack or require an additional authentication step that involves another device. Such defense strategies are either not strong enough or lower the usability of the system. Based on the fact that legitimate voice commands should only come from humans rather than a playback device, we propose a novel defense strategy that is able to detect the sound source of a voice command based on its acoustic features. The proposed defense strategy does not require any information other than the voice command itself and can protect a system from multiple types of spoofing attacks. Our proof-of-concept experiments verify the feasibility and effectiveness of this defense strategy.

ReMASC: Realistic Replay Attack Corpus for Voice Controlled Systems

“…In order to defend against these attacks, the work presented in [9] and [10] each proposes a defense strategy to protect VCSs by identifying the sound source of the received voice commands and rejecting those that are not from a human speaker, merely by analyzing the acoustic cues within the voice commands. This approach is based on the observation that legitimate voice commands should only come from human speakers rather than a playback device and that attacks such as self-triggered attacks, hidden voice commands, and audio adversarial examples rely on a playback device.…”

“…That is, we are able to leverage the differences in the sound production mechanisms of humans and playback devices, which lead to differences in the frequencies and the directivity of the output voice signal. For example, in [9], the authors leverage the presence of significant low-frequency signals to distinguish electronic speakers from human speakers, while in [10], the authors use a combination of features includ- 1 Data available at https://github.com/yuangongnd/remasc ing fundamental frequency and Mel-frequency cepstral coefficients (MFCCs), and propose a data-driven approach.…”

“…The key idea in [9,10] is actually an extension of the antispoofing technologies used for protecting automatic speaker verification (ASV) systems. Many prior efforts (such as presented in [11,12,13,14,15,16,17,18,19,20,21,22]) have attempted to differentiate between original and replayed speech using the RedDots Replayed data set [23].…”

“…Therefore, since the RedDots Replayed data set has been recorded with short speaker-device distances, in indoor environments, and with devices that use a single microphone, it is not a suitable choice for research on VCS protection. Consequently, prior VCS protection research [9,10] relied on self-collected non-public data for their experiments, where these data sets typically contain samples from a small number of subjects (≤ 8), limited environmental settings and number of voice commands (≤ 8), and only in single-microphone settings. The 2019 ASVspoof challenge [24] provides simulated data for clear theoretical analysis of audio spoofing attacks in physical environments, but also leaves the simulation-to-reality gap.…”

“…The 2019 ASVspoof challenge [24] provides simulated data for clear theoretical analysis of audio spoofing attacks in physical environments, but also leaves the simulation-to-reality gap. Therefore, in order to facilitate future research on the protection of VCSs, we present the ReMASC (Realistic Replay Attack Microphone Array Speech Corpus) data set, which, compared to the RedDots Replayed data set and the data sets used in [9,10], has more data variety and is closer to realistic VCS usage scenarios and settings. Specifically, the data set contains recordings from 50 subjects of both genders and with different ages and accents.…”