Study of Pre-Processing Defenses Against Adversarial Attacks on State-of-the-Art Speaker Recognition Systems

Joshi, Sonal; Villalba, Jesús; Żelasko, Piotr; Moro-Velázquez, Laureano; Dehak, Najim

doi:10.1109/tifs.2021.3116438

Cited by 33 publications

(19 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our experiment of adversarial training, we adversarially train Kaldi (the aforementioned Mini LibriSpeech model) on the Mini LibriSpeech dataset 14 . We set 𝑝 to ∞, iterations to 10, and the step size 𝛼 to 𝜖/5 for PGD, which are the same with [50,51]. Since the prior work [51] demonstrates that adversarial training with 𝜖 > 0.01 will be not able to converge, we set 𝜖 to 0.002, 0.004 and 0.006.…”

Section: H Results Of Possible Countermeasuresmentioning

confidence: 99%

“…We set 𝑝 to ∞, iterations to 10, and the step size 𝛼 to 𝜖/5 for PGD, which are the same with [50,51]. Since the prior work [51] demonstrates that adversarial training with 𝜖 > 0.01 will be not able to converge, we set 𝜖 to 0.002, 0.004 and 0.006. For evaluation, we also generate 10 audio AEs from Mini LibriSpeech model.…”

Section: H Results Of Possible Countermeasuresmentioning

confidence: 99%

“…However, while the SRoA can be reduced by 70%∼90%, it also inevitably brings a significant performance degradation, i.e., a 10%∼20% accuracy loss, and such loss is unacceptable for commercial ASR systems. Moreover, adversarial training will significantly increase the training time and costs [51,76], particularly for ASR systems. For example, for the Mini LibriSpeech dataset containing only 5 hours of audio data, it took about 10 days for us to adversarially train Kaldi model on six NVIDIA 2080Ti GPUs, while commonly-used voice datasets, like LibriSpeech and Common Voice, have around 1000 hours of voice data), making it almost impractical on large-scale models and datasets.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information

Zheng

Jiang

Wang

et al. 2021

Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

Adversarial attacks against commercial black-box speech platforms, including cloud speech APIs and voice control devices, have received little attention until recent years. Constructing such attacks is difficult mainly due to the unique characteristics of time-domain speech signals and the much more complex architecture of acoustic systems. The current "black-box" attacks all heavily rely on the knowledge of prediction/confidence scores or other probability information to craft effective adversarial examples (AEs), which can be intuitively defended by service providers without returning these messages. In this paper, we take one more step forward and propose two novel adversarial attacks in more practical and rigorous scenarios. For commercial cloud speech APIs, we propose Occam, a decision-only black-box adversarial attack, where only final decisions are available to the adversary. In Occam, we formulate the decision-only AE generation as a discontinuous large-scale global optimization problem, and solve it by adaptively decomposing this complicated problem into a set of sub-problems and cooperatively optimizing each one. Our Occam is a one-size-fits-all approach, which achieves 100% success rates of attacks (SRoA) with an average SNR of 14.23dB, on a wide range of popular speech and speaker recognition APIs, including Google, Alibaba, Microsoft, Tencent, iFlytek, and Jingdong, outperforming the state-of-the-art black-box attacks. For commercial voice control devices, we propose NI-Occam, the first non-interactive physical adversarial attack, where the adversary does not need to query the oracle and has no access to its internal information and training data. We, for the first time, combine adversarial attacks with model inversion attacks, and thus generate the physically-effective audio AEs with high transferability without any interaction with target devices. Our experimental results show that NI-Occam can successfully fool Apple Siri, Microsoft Cortana, Google Assistant, iFlytek and Amazon Echo with an average SRoA of 52% and SNR of 9.65dB, shedding light on non-interactive physical attacks against voice control devices.

show abstract

Section: H Results Of Possible Countermeasuresmentioning

confidence: 99%

Section: H Results Of Possible Countermeasuresmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information

Zheng

Jiang

Wang

et al. 2021

Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

show abstract

“…In general, the modules developed for video and audio interaction represented the state-of-the-art at the time of the development of the mirror. More robust methods for both video and audio identification were recently proposed [74,75]. At the same time, more effective face attribute estimation [76] and speaker emotion recognition [77,78] methods were presented.…”

Section: Discussionmentioning

confidence: 99%

A Smart Mirror for Emotion Monitoring in Home Environments

Bianco

Celona

Ciocca

et al. 2021

Sensors

View full text Add to dashboard Cite

Smart mirrors are devices that can display any kind of information and can interact with the user using touch and voice commands. Different kinds of smart mirrors exist: general purpose, medical, fashion, and other task specific ones. General purpose smart mirrors are suitable for home environments but the exiting ones offer similar, limited functionalities. In this paper, we present a general-purpose smart mirror that integrates several functionalities, standard and advanced, to support users in their everyday life. Among the advanced functionalities are the capabilities of detecting a person’s emotions, the short- and long-term monitoring and analysis of the emotions, a double authentication protocol to preserve the privacy, and the integration of Alexa Skills to extend the applications of the smart mirrors. We exploit a deep learning technique to develop most of the smart functionalities. The effectiveness of the device is demonstrated by the performances of the implemented functionalities, and the evaluation in terms of its usability with real users.

show abstract

“…Most of the countermeasures in the literature can be primarily classified into two types-proactive and reactive [6,7]. Proactive countermeasures focus on making the system under attack (a.k.a.…”

Section: Introductionmentioning

confidence: 99%

AdvEst: Adversarial Perturbation Estimation to Classify and Detect Adversarial Attacks against Speaker Identification

Joshi¹,

Kataria²,

Villalba³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Adversarial attacks pose a severe security threat to the state-ofthe-art speaker identification systems, thereby making it vital to propose countermeasures against them. Building on our previous work that used representation learning to classify and detect adversarial attacks, we propose an improvement to it using Ad-vEst, a method to estimate adversarial perturbation. First, we prove our claim that training the representation learning network using adversarial perturbations as opposed to adversarial examples (consisting of the combination of clean signal and adversarial perturbation) is beneficial because it eliminates nuisance information. At inference time, we use a time-domain denoiser to estimate the adversarial perturbations from adversarial examples. Using our improved representation learning approach to obtain attack embeddings (signatures), we evaluate their performance for three applications: known attack classification, attack verification, and unknown attack detection. We show that common attacks in the literature (Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), Carlini-Wagner (CW) with different Lp threat models) can be classified with an accuracy of ∼ 96%. We also detect unknown attacks with an equal error rate (EER) of ∼9%, which is absolute improvement of ∼12% from our previous work.

show abstract

Study of Pre-Processing Defenses Against Adversarial Attacks on State-of-the-Art Speaker Recognition Systems

Cited by 33 publications

References 40 publications

Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information

Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information

A Smart Mirror for Emotion Monitoring in Home Environments

AdvEst: Adversarial Perturbation Estimation to Classify and Detect Adversarial Attacks against Speaker Identification

Contact Info

Product

Resources

About