Adversarial Black-Box Attacks on Automatic Speech Recognition Systems Using Multi-Objective Evolutionary Optimization

Khare, Shreya; Aralikatte, Rahul; Mani, Senthil

doi:10.21437/interspeech.2019-2420

Cited by 35 publications

(9 citation statements)

References 12 publications

(18 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though the success of adversarial attack on image recognition systems has been ported to the speech recognition systems in both the white-box setting (e.g., [8], [9]) and black-box setting (e.g., [10], [11]), relatively little research has been done on SRSs. Essentially, the speech signal of an utterance consists of two major parts: the underlying text and the characteristics of the speaker.…”

Section: Introductionmentioning

confidence: 99%

Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems

Chen

Chenb

Fan

et al. 2021

2021 IEEE Symposium on Security and Privacy (SP)

114

144

View full text Add to dashboard Cite

Speaker recognition (SR) is widely used in our daily life as a biometric authentication or identification mechanism. The popularity of SR brings in serious security concerns, as demonstrated by recent adversarial attacks. However, the impacts of such threats in the practical black-box setting are still open, since current attacks consider the white-box setting only.In this paper, we conduct the first comprehensive and systematic study of the adversarial attacks on SR systems (SRSs) to understand their security weakness in the practical blackbox setting. For this purpose, we propose an adversarial attack, named FAKEBOB, to craft adversarial samples. Specifically, we formulate the adversarial sample generation as an optimization problem, incorporated with the confidence of adversarial samples and maximal distortion to balance between the strength and imperceptibility of adversarial voices. One key contribution is to propose a novel algorithm to estimate the score threshold, a feature in SRSs, and use it in the optimization problem to solve the optimization problem. We demonstrate that FAKEBOB achieves close to 100% targeted attack success rate on both open-source and commercial systems. We further demonstrate that FAKEBOB is also effective (at least 65% untargeted success rate) on both open-source and commercial systems when playing over the air in the physical world. Moreover, we have conducted a human study which reveals that it is hard for human to differentiate the speakers of the original and adversarial voices. Last but not least, we show that three promising defense methods for adversarial attack from the speech recognition domain become ineffective on SRSs against FAKEBOB, which calls for more effective defense methods. We highlight that our study peeks into the security implications of adversarial attacks on SRSs, and realistically fosters to improve the security robustness of SRSs.

show abstract

Section: Introductionmentioning

confidence: 99%

Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems

Chen

Chenb

Fan

et al. 2021

2021 IEEE Symposium on Security and Privacy (SP)

114

144

View full text Add to dashboard Cite

show abstract

“…This is the third class of hidden commands, the adversarial command. There have been extensive studies in the speech adversarial commands domain and we review representative works, including Iter'17 [35], Alzantot'18 [78], Carlini'18 [36], CommanderSong'18 [37], Taori'18 [79], Khare'19 [80], Schönherr'19 [9], Yakura'19 [81], Qin'19 [82], Szurley'19 [83], Imperio'19 [84], Yang'19 [85], Metamorph'20 [86], and AdvPulse'20 [87].…”

Section: B Hidden Voice Commandsmentioning

confidence: 99%

“…Khare'19 [80]: It proposes a new framework using multiobjective evolutionary optimization to generate adversarial commands for both nontargeted and targeted attacks on black-box ASRs. For the generation process, a set of original audio inputs is selected and random uniform noise is added to them as the initialization.…”

Section: B Hidden Voice Commandsmentioning

confidence: 99%

Personal Voice Assistant Security and Privacy—A Survey

Cheng

Roedig

2022

Proc. IEEE

View full text Add to dashboard Cite

Personal voice assistants (PVAs) are increasingly used as interfaces to digital environments. Voice commands are used to interact with phones, smart homes, or cars. In the United States alone, the number of smart speakers, such as Amazon's Echo and Google Home, has grown by 78% to 118.5 million, and 21% of the U.S. population own at least one device. Given the increasing dependency of society on PVAs, security and privacy of these have become a major concern of users, manufacturers, and policy makers. Consequently, a steep increase in research efforts addressing security and privacy of PVAs can be observed in recent years. While some security and privacy research applicable to the PVA domain predates their recent increase in popularity, many new research strands have emerged. This article provides a survey of the state of the art in PVA security and privacy. The focus of this work is on the security and privacy challenges arising from the use of the acoustic channel. Work that describes both attacks and countermeasures is discussed. We highlight established areas such as voice authentication (VA) and new areas such as acoustic Denial of Service (DoS) that deserve more attention. This survey describes research areas where the threat is relatively well understood but where countermeasures are lacking, for example, in the area of hidden voice commands. We also discuss work that looks at privacy implications; for example, work on management of recording Manuscript

show abstract

“…Recently, some adversarial example‐based test case generation approaches for black‐box speech recognition systems [20,21,24,25] have been proposed. However, these approaches still suffer from the following limitations: Most approaches focus on generating non‐targeted adversarial examples , which are incapable of generating adversarial examples for target text. Most of the studies use relatively monotonous experimental speech datasets composed of simple words and phrases .…”

Section: Introductionmentioning

confidence: 99%

“…In the sense of fully testing the performance of the system under the interference of targeted adversarial examples, these special adversarial examples generated for black-box speech recognition systems have more practical value. Recently, some adversarial example-based test case generation approaches for black-box speech recognition systems [20,21,24,25] have been proposed. However, these approaches still suffer from the following limitations:…”

mentioning

confidence: 99%

Adversarial example‐based test case generation for black‐box speech recognition systems

Cai

Zhang

Dong

et al. 2023

Software Testing Verif & Rel

View full text Add to dashboard Cite

Test case generation techniques based on adversarial examples are commonly used to enhance the reliability and robustness of image-based and text-based machine learning applications. However, efficient techniques for speech recognition systems are still absent. This paper proposes a family of methods that generate targeted adversarial examples for speech recognition systems. All are based on the firefly algorithm (F), and are enhanced with gauss mutations and / or gradient estimation (F-GM, F-GE, F-GMGE) to fit the specific problem of targeted adversarial test case generation. We conduct an experimental evaluation on three different types of speech datasets, including Google Command, Common Voice and LibriSpeech. In addition, we recruit volunteers to evaluate the performance of the adversarial examples. The experimental results show that, compared with existing approaches, these approaches can effectively improve the success rate of the targeted adversarial example generation. The code is publicly available at https://github.com/HanboCai/FGMGE.

show abstract

Adversarial Black-Box Attacks on Automatic Speech Recognition Systems Using Multi-Objective Evolutionary Optimization

Cited by 35 publications

References 12 publications

Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems

Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems

Personal Voice Assistant Security and Privacy—A Survey

Adversarial example‐based test case generation for black‐box speech recognition systems

Contact Info

Product

Resources

About