Robust Speaker Recognition Using Speech Enhancement And Attention Model

Abstract

In this paper, a novel architecture for speaker recognition is proposed by cascading speech enhancement and speaker processing. It aims to improve speaker recognition performance when speech signals are corrupted by noise. Instead of separately processing speech enhancement and speaker recognition, the two modules are integrated into one framework by a joint optimisation using deep neural networks. Furthermore, to increase the robustness against noise, a multi-stage attention mechanism is employed to highlight the speaker related features learned from context information in both time and frequency domains. To evaluate speaker identification and verification performance of the proposed approach, VoxCeleb1, one of mostly used benchmark datasets, is used. Moreover, the robustness evaluation is also conducted on VoxCeleb1 when its being corrupted by three types of interferences, general noise, music, and babble, at different signal-to-noise ratio (SNR) levels. The obtained results show that the proposed approach using speech enhancement and multi-stage attention models outperforms two strong baselines in different acoustic conditions in our experiments.

Cam: Context-Aware Masking for Robust Speaker Verification

“…In [14], researchers evaluate and optimize the speech enhancement model based on perceptual loss, which is calculated by a pre-trained speaker embedding network. In [15,16,17], researchers connect and train the speech enhancement and speaker embedding networks in an end-to-end manner. Both perceptual loss and endto-end training optimize the speech enhancement network with the target of improving speaker verification performance instead of decreasing the regression error between enhanced and clean features.…”

Task-specific Optimization of Virtual Channel Linear Prediction-based Speech Dereverberation Front-End for Far-Field Speaker Verification

“…Particularly, in a single-microphone setup, a speech enhancement front-end may lead to performance degradation of ASV systems unless carefully designed or optimized [8]- [10]. Therefore, several task-specific optimization (TSO) methods have been proposed to build DNN-based singlechannel front-ends [9]- [11], wherein the DNNs are trained for acoustic feature enhancement [9] or spectral masking [10] by a task-specific method, by exploiting a pretrained DSE model. In a similar context, in [11], a denoising DNN was jointly optimized with a pretrained DSE model.…”

“…Therefore, several task-specific optimization (TSO) methods have been proposed to build DNN-based singlechannel front-ends [9]- [11], wherein the DNNs are trained for acoustic feature enhancement [9] or spectral masking [10] by a task-specific method, by exploiting a pretrained DSE model. In a similar context, in [11], a denoising DNN was jointly optimized with a pretrained DSE model. Concurrently, the standard acoustic features were substituted by the bottleneck features learned from a DNN, which was adversarially trained to classify noise types [12].…”

“…Specifically, we present a novel front-end design based on the recently proposed virtual acoustic channel expansion (VACE)-WPE [36], [37], which utilizes a dual-channel neural WPE [38] in a single-microphone setup and a virtual signal generated by a DNN as the secondary input. Similar to [6], [9]- [11], we apply a TSO framework such that given a pretrained VACE-WPE and a pretrained DSE model, the VACE-WPE is additionally fine-tuned to reduce the difference between a processed signal and a "noise-free" target signal in the embedding space of the DSE model. Herein, the pretraining of a front-end is conducted using "noisy" but reverberation-free target signals, which is a key difference from the previous studies that applied a TSO framework [9]- [11].…”

“…Similar to [6], [9]- [11], we apply a TSO framework such that given a pretrained VACE-WPE and a pretrained DSE model, the VACE-WPE is additionally fine-tuned to reduce the difference between a processed signal and a "noise-free" target signal in the embedding space of the DSE model. Herein, the pretraining of a front-end is conducted using "noisy" but reverberation-free target signals, which is a key difference from the previous studies that applied a TSO framework [9]- [11]. In the subsequent TSO stage, a "noise-free" target signal is employed based on the denoising capability of the VACE-WPE, a new property of the VACE-WPE demonstrated experimentally in this study.…”