Language Normalization for Bilingual Speaker Recognition Systems

Abstract

In this study, we focus on the problem of removing/normalizing the impact of spoken language variation in Bilingual Speaker Recognition (BSR) systems. In addition to environment, recording, and channel mismatches, spoken language mismatch is an additional factor resulting in performance degradation in speaker recognition systems. In today's world, the number of bilingual speakers is increasing with English becoming the universal second language. Data sparseness is becoming an important research issue to deploy speaker recognition systems with limited resources (e.g., short train/test durations). Therefore, leveraging existing resources from different languages becomes a practical concern in limited-resource BSR applications, and effective language normalization schemes are required to achieve more robust speaker recognition systems. Here, we propose two novel algorithms to address the spoken language mismatch problem: normalization at the utterance-level via Language Identication (LID), and normalization at the segment-level via multilingual Phone Recognition (PR). We evaluated our algorithms using a bilingual (Spanish-English) speaker set of 80 speakers. Experimental results show improvements over a baseline system which employs fusion of language-dependent speaker models with xed weights.

An overview of robustness related issues in speaker recognition

“…In [73], authors proposed two novel algorithms towards spoken language mismatch problem: the first algorithm merges language-dependent system outputs by using Language Identification (LID) scores of each utterance as fusion weights. And in the second algorithm, fusion is done at the segment-level via multilingual Phone Recognition (PR).…”

Cross-lingual speaker verification based on linear transform

“…Factor analysis treats the language as a latent variable and marginalizes it out when conducting verification. For example, the method proposed in [6] infers the posterior probabilities of a test speech belonging to different languages. With these posteriors, the score of the test speech against the claimed speaker is computed as the fusion (summation) of the scores tested against the models of the speaker enrolled with each language.…”

Türkçe Konuşmaci Doğrulamada Di̇l Uyumsuzluğunun Etki̇si̇

“…Another solution for bilingual speaker recognition is training two separate speaker models for each target speaker one with Spanish data and the other using English data (Akbacak and Hansen, 2007). During the recognition phase, first a language detector is used to detect the language of test utterance for choosing the correct speaker model (Akbacak and Hansen, 2007). However, both of these two proposed solutions require knowledge about the languages of training and test utterances.…”

“…To alleviate this degradation, authors proposed to model each speaker using both languages (Ma and Meng, 2004). Another solution for bilingual speaker recognition is training two separate speaker models for each target speaker one with Spanish data and the other using English data (Akbacak and Hansen, 2007). During the recognition phase, first a language detector is used to detect the language of test utterance for choosing the correct speaker model (Akbacak and Hansen, 2007).…”