Adversarial attack and defense strategies for deep speaker recognition systems

Jati, Arindam; Hsu, Chin-Cheng; Pal, Mahesh; Peri, Raghuveer; AbdAlmageed, Wael; Narayanan, Shrikanth

doi:10.1016/j.csl.2021.101199

Cited by 53 publications

(64 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This category shows more resistance to voice synthesis and conversion spoofing attacks, but it is valuable to human impersonation. Employing this feature in speaker recognition systems provides robustness to adversarial attacks to which most deep speaker recognition systems are prone to [ 40 ]. One can argue that a speaker recognition system that employs short-term spectral and prosodic features is more robust than those systems that employ only one type of these features.…”

Section: Related Workmentioning

confidence: 99%

A Two-Level Speaker Identification System via Fusion of Heterogeneous Classifiers and Complementary Feature Cooperation

Al-Qaderi

Lahamer

Rad

2021

Sensors

View full text Add to dashboard Cite

We present a new architecture to address the challenges of speaker identification that arise in interaction of humans with social robots. Though deep learning systems have led to impressive performance in many speech applications, limited speech data at training stage and short utterances with background noise at test stage present challenges and are still open problems as no optimum solution has been reported to date. The proposed design employs a generative model namely the Gaussian mixture model (GMM) and a discriminative model—support vector machine (SVM) classifiers as well as prosodic features and short-term spectral features to concurrently classify a speaker’s gender and his/her identity. The proposed architecture works in a semi-sequential manner consisting of two stages: the first classifier exploits the prosodic features to determine the speaker’s gender which in turn is used with the short-term spectral features as inputs to the second classifier system in order to identify the speaker. The second classifier system employs two types of short-term spectral features; namely mel-frequency cepstral coefficients (MFCC) and gammatone frequency cepstral coefficients (GFCC) as well as gender information as inputs to two different classifiers (GMM and GMM supervector-based SVM) which in total leads to construction of four classifiers. The outputs from the second stage classifiers; namely GMM-MFCC maximum likelihood classifier (MLC), GMM-GFCC MLC, GMM-MFCC supervector SVM, and GMM-GFCC supervector SVM are fused at score level by the weighted Borda count approach. The weight factors are computed on the fly via Mamdani fuzzy inference system that its inputs are the signal to noise ratio and the length of utterance. Experimental evaluations suggest that the proposed architecture and the fusion framework are promising and can improve the recognition performance of the system in challenging environments where the signal-to-noise ratio is low, and the length of utterance is short; such scenarios often arise in social robot interactions with humans.

show abstract

Section: Related Workmentioning

confidence: 99%

A Two-Level Speaker Identification System via Fusion of Heterogeneous Classifiers and Complementary Feature Cooperation

Al-Qaderi

Lahamer

Rad

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…f (•) is a well-trained classifier. The authors [78] provide a wide range of investigation related to the topic and pointed out such a strategy can largely infect the speaker recognition methods. However, adversarial training can considerably defend such perturbation attacks [79].…”

Section: Figure 16mentioning

confidence: 99%

“…• Tamper proof: Speaker recognition systems should be tamper-proof against adversarial attacks and voice mimicry. In some cases, adversarial attacks require extensive investigation over the available architectures resulting in hard to generate [78]. However, voice mimicry does not significantly affect speaker recognition models [126].…”

Section: Metamentioning

confidence: 99%

Deep Speaker Recognition: Process, Progress, and Challenges

et al. 2021

View full text Add to dashboard Cite

Speaker recognition is related to human biometrics dealing with the identification of speakers from their speech. Speaker recognition is an active research area and being widely investigated using artificially intelligent mechanisms. Though speaker recognition systems were previously constructed using handcrafted statistical means of machine learning, currently it is being shifted to state-of-the-art deep learning strategies. Further, deep learning being a fast-paced domain, an absence of comprehensive survey is observed in the current deep speaker recognition technologies. In this paper, we focus on deep speaker recognition technologies. The paper particularly introduces a taxonomy, explains the progress, architectural strategies and processes of some distinctive approaches. Further, the manuscript classifies and enlists the currently available datasets and programming tools. Finally, the paper investigates the challenges and future directives of deep speaker recognition technology.

show abstract

“…As a result of the rapid development of modern speech processing technology, people can easily edit "false" audio that cannot be easily distinguished by hearing, which makes it more difficult for people to distinguish the true and false audio in social networks. Therefore, the application of machine learning methods to mine hidden information in acoustic signals to identify authenticity and monitor risks [4,5] has become a research hotspot in the field of acoustic signal processing. The special complex noise and sudden acoustic events in ambient background sounds are some of the important factors to judge audio authenticity.…”

Section: Introductionmentioning

confidence: 99%

Audio Information Camouflage Detection for Social Networks

Lou

Zuo

et al. 2021

Front. Phys.

View full text Add to dashboard Cite

Sending camouflaged audio information for fraud in social networks has become a new means of social networks attack. The hidden acoustic events in the audio scene play an important role in the detection of camouflaged audio information. Therefore, the application of machine learning methods to represent hidden information in audio streams has become a hot issue in the field of network security detection. This study proposes a heuristic mask for empirical mode decomposition (HM-EMD) method for extracting hidden features from audio streams. The method consists of two parts: First, it constructs heuristic mask signals related to the signal’s structure to solve the modal mixing problem in intrinsic mode function (IMF) and obtains a pure IMF related to the signal’s structure. Second, a series of hidden features in environment-oriented audio streams is constructed on the basis of the IMF. A machine learning method and hidden information features are subsequently used for audio stream scene classification. Experimental results show that the hidden information features of audio streams based on HM-EMD are better than the classical mel cepstrum coefficients (MFCC) under different classifiers. Moreover, the classification accuracy achieved with HM-EMD increases by 17.4 percentage points under the three-layer perceptron and by 1.3% under the depth model of TridentResNet. The hidden information features extracted by HM-EMD from audio streams revealed that the proposed method could effectively detect camouflaged audio information in social networks, which provides a new research idea for improving the security of social networks.

show abstract

Adversarial attack and defense strategies for deep speaker recognition systems

Cited by 53 publications

References 6 publications

A Two-Level Speaker Identification System via Fusion of Heterogeneous Classifiers and Complementary Feature Cooperation

A Two-Level Speaker Identification System via Fusion of Heterogeneous Classifiers and Complementary Feature Cooperation

Deep Speaker Recognition: Process, Progress, and Challenges

Audio Information Camouflage Detection for Social Networks

Contact Info

Product

Resources

About