DEAAN: Disentangled Embedding and Adversarial Adaptation Network for Robust Speaker Representation Learning

Abstract

Despite speaker verification has achieved significant performance improvement with the development of deep neural networks, domain mismatch is still a challenging problem in this field. In this study, we propose a novel framework to disentangle speaker-related and domain-specific features and apply domain adaptation on the speaker-related feature space solely. Instead of performing domain adaptation directly on the feature space where domain information is not removed, using disentanglement can efficiently boost adaptation performance. To be specific, our model's input speech from the source and target domains is first encoded into different latent feature spaces. The adversarial domain adaptation is conducted on the shared speaker-related feature space to encourage the property of domain-invariance. Further, we minimize the mutual information between speaker-related and domain-specific features for both domains to enforce the disentanglement. Experimental results on the VOiCES dataset demonstrate that our proposed framework can effectively generate more speaker-discriminative and domain-invariant speaker representations with a relative 20.3% reduction of EER compared to the original ResNet-based system.

A Survey on Text-Dependent and Text-Independent Speaker Verification

“…To address this problem, the authors of [53], [54] minimized the maximum mean discrepancy (MMD) [55] across different languages to create a language-invariant speaker embedding space. Besides, domain adversarial training [56] has been successfully applied [57]- [61] to produce language-invariant speaker embeddings. There are also DA methods that directly adapt the PLDA covariance matrices to match the target distribution, e.g., CORAL+ [62] and Kaldi's PLDA adaptation [63].…”

Leveraging Speaker Embeddings with Adversarial Multi-task Learning for Age Group Classification

“…Recently, speaker embeddings learned by deep neural network (DNN)-based architectures such as x-vector and ResNet have shown more impressive performance on SR than the previous state-of-the-art method, the i-vector-based approach [2]. The DNN-based approach can extract speaker-discriminative and robust speaker embeddings by training on various utterances from a large-scale SR dataset [3]. However, DNN-based speaker-discriminative embeddings do not represent the speech feature itself and cannot be effectively used as the feature vector to analyze speech.…”

Disentangled Speaker Representation Learning via Mutual Information Minimization

“…For instance, joint factor embedding (JFE) [18] framework simultaneously extracts speaker and nuisance (i.e., non-speaker) embeddings and maximizes entropy (or uncertainty) on their opposite task, while minimizing correlation between two embeddings using mean absolute Pearson's correlation (MAPC) computed batch-wise. Similarly, [19], [20] divided features into the speaker and residual embeddings and increased their uncertainty on the contrary task, and [21] minimized mutual information via mutual information neural estimator (MINE) with GRL. Additionally, they adopted an autoencoder framework for training merged embedding to maintain the complete information of input speech [19]- [21].…”

“…Similarly, [19], [20] divided features into the speaker and residual embeddings and increased their uncertainty on the contrary task, and [21] minimized mutual information via mutual information neural estimator (MINE) with GRL. Additionally, they adopted an autoencoder framework for training merged embedding to maintain the complete information of input speech [19]- [21]. However, naively increasing uncertainty on the other task does not guarantee disentanglement.…”