State-of-the-Art Speaker Recognition for Telephone and Video Speech: The JHU-MIT Submission for NIST SRE18

Villalba, Jesús; Chen, Nanxin; Snyder, David; Garcia‐Romero, Daniel; McCree, Alan; Sell, Gregory; Borgstrom, Jonas; Richardson, F. S.; Shon, Suwon; Grondin, François; Dehak, Réda; García-Perera, Leibny Paola; Povey, Daniel; Torres-Carrasquillo, Pedro A.; Khudanpur, Sanjeev; Dehak, Najim

doi:10.21437/interspeech.2019-2713

Cited by 78 publications

(60 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, we explain the transfer learning approach followed to perform SER. It is shown in the literature that i-vectors [17], speaker diarization [18,19,20,21]. In this work, we only exploit the x-vector model because of its superiority over i-vectors [22] and also because it is easy to adapt for down-stream tasks.…”

Section: Our Approachmentioning

confidence: 99%

See 1 more Smart Citation

X-Vectors Meet Emotions: A Study On Dependencies Between Emotion and Speaker Recognition

Pappagari

Wang

Villalba

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

In this work, we explore the dependencies between speaker recognition and emotion recognition. We first show that knowledge learned for speaker recognition can be reused for emotion recognition through transfer learning. Then, we show the effect of emotion on speaker recognition. For emotion recognition, we show that using a simple linear model is enough to obtain good performance on the features extracted from pre-trained models such as the x-vector model. Then, we improve emotion recognition performance by finetuning for emotion classification. We evaluated our experiments on three different types of datasets: IEMOCAP, MSP-Podcast, and Crema-D. By fine-tuning, we obtained 30.40%, 7.99%, and 8.61% absolute improvement on IEMOCAP, MSP-Podcast, and Crema-D respectively over baseline model with no pre-training. Finally, we present results on the effect of emotion on speaker verification. We observed that speaker verification performance is prone to changes in test speaker emotions. We found that trials with angry utterances performed worst in all three datasets. We hope our analysis will initiate a new line of research in the speaker recognition community.

show abstract

Section: Our Approachmentioning

confidence: 99%

“…In this paper, we used state-of-the-art ResNet x-vector model reported in [17] for utterance level speaker embedding extraction. The network consisted of three parts: frame-level representation learning network, pooling network, and utterance-level classifier.…”

Section: X-vector Modelmentioning

confidence: 99%

X-Vectors Meet Emotions: A Study On Dependencies Between Emotion and Speaker Recognition

Pappagari

Wang

Villalba

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are three components in a typical end-to-end speaker recognition system: an encoder network, a statistical pooling layer, and a classifier [16]. An encoder network acts as a frame-level feature extractor, the statistical pooling layer summarizes frame-level representations to a fixed-dimensional utterance-level embedding, and the classifier determines the speaker identity based on the embedding.…”

Section: Neural Speaker Embeddingsmentioning

confidence: 99%

“…One aspect of our study is therefore an attempt to find out how effective these recent developments in speaker verification are for speaker adaption in TTS. More specifically we investigate the capability of neural speaker embeddings [16,17,19] to capture and model characteristics of speakers that were unseen during TTS model training. For this purpose, we extend an improved Tacotron system in [28] to a multi-speaker TTS system and conduct systematic analysis to answer the above question.…”

Section: Introductionmentioning

confidence: 99%

Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings

Cooper

Lai

Yasuda

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

146

125

View full text Add to dashboard Cite

While speaker adaptation for end-to-end speech synthesis using speaker embeddings can produce good speaker similarity for speakers seen during training, there remains a gap for zero-shot adaptation to unseen speakers. We investigate multi-speaker modeling for end-to-end text-to-speech synthesis and study the effects of different types of state-of-the-art neural speaker embeddings on speaker similarity for unseen speakers. Learnable dictionary encoding-based speaker embeddings with angular softmax loss can improve equal error rates over x-vectors in a speaker verification task; these embeddings also improve speaker similarity and naturalness for unseen speakers when used for zero-shot adaptation to new speakers in endto-end speech synthesis.

show abstract

“…Text independent speaker verification aims to verify whether an utterance is pronounced by a hypothesized speaker according to his/her pre-recorded utterances without limiting the speech contents. The state-of-the-art text-independent speaker verification systems [1][2][3][4] use deep neural networks (DNNs) to project speech recordings with different lengths into a common low dimensional embedding space where the speakers' identities are represented. Such a method is called deep embedding, where the embedding networks have three key components-network structure [1,3,[5][6][7], pooling layer [1,[8][9][10][11][12], and loss function [13][14][15][16][17].…”

Section: Introductionmentioning

confidence: 99%

Partial AUC Optimization Based Deep Speaker Embeddings with Class-Center Learning for Text-Independent Speaker Verification

Bai

Zhang

Chen

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Deep embedding based text-independent speaker verification has demonstrated superior performance to traditional methods in many challenging scenarios. Its loss functions can be generally categorized into two classes, i.e., verification and identification. The verification loss functions match the pipeline of speaker verification, but their implementations are difficult. Thus, most state-of-the-art deep embedding methods use the identification loss functions with softmax output units or their variants. In this paper, we propose a verification loss function, named the maximization of partial area under the Receiver-operating-characteristic (ROC) curve (pAUC), for deep embedding based text-independent speaker verification. We also propose a class-center based training trial construction method to improve the training efficiency, which is critical for the proposed loss function to be comparable to the identification loss in performance. Experiments on the Speaker in the Wild (SITW) and NIST SRE 2016 datasets show that the proposed pAUC loss function is highly competitive with the state-of-the-art identification loss functions.

show abstract

State-of-the-Art Speaker Recognition for Telephone and Video Speech: The JHU-MIT Submission for NIST SRE18

Cited by 78 publications

References 13 publications

X-Vectors Meet Emotions: A Study On Dependencies Between Emotion and Speaker Recognition

X-Vectors Meet Emotions: A Study On Dependencies Between Emotion and Speaker Recognition

Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings

Partial AUC Optimization Based Deep Speaker Embeddings with Class-Center Learning for Text-Independent Speaker Verification

Contact Info

Product

Resources

About