Improving N-gram language modeling for code-switching speech recognition

Code-switching is about dealing with alternative languages in speech or text. It is partially speaker-dependent and domainrelated, so completely explaining the phenomenon by linguistic rules is challenging. Compared to most monolingual tasks, insufficient data is an issue for code-switching. To mitigate the issue without expensive human annotation, we proposed an unsupervised method for code-switching data augmentation. By utilizing a generative adversarial network, we can generate intra-sentential code-switching sentences from monolingual sentences. We applied the proposed method on two corpora, and the result shows that the generated code-switching sentences improve the performance of code-switching language models.

Section: Resultsmentioning

confidence: 99%

Code-Switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentation

Chang¹,

Chuang²,

Lee³

2019

“…Existing solutions leverage on using linguistic information to generalize word lexicon. Many incorporate class [4,5,6,7], Part-of-Speech [8,9] or language ID [5] together with word input to improve generalization of the language model to the unseen test sequence. In [10,11,12] code-switch permission constraints are used to provide a code-switch probability for the language model.…”

Section: Introductionmentioning

confidence: 99%

Linguistically Motivated Parallel Data Augmentation for Code-Switch Language Modeling

Lee¹,

Yue²,

Li³

2019

Self Cite

Code-switch language modeling is challenging due to data scarcity as well as expanded vocabulary that involves two languages. We present a novel computational method to generate synthetic code-switch data using the Matrix Language Frame theory to alleviate the issue of data scarcity. The proposed method makes use of augmented parallel data to supplement the real code-switch data. We use the synthetic data to pre-train the language model. We show that the pre-trained language model can match the performance of vanilla models when it is finetuned with 2.5 times less real code-switch data. We also show that the perplexity of a RNN based language model pre-trained on synthetic code-switch data and fine-tuned with real codeswitch data is significantly lower than that of the model trained on real code-switch data alone and the reduction in perplexity translates into 1.45% absolute reduction in WER in a speech recognition experiment.

“…Code-switching (CS), the alternating use of two or more languages in a single conversation, is a common phenomenon in multilingual communities. There is increasing research interest in developing CS automatic speech recognition (ASR) systems [1][2][3][4][5][6][7][8][9][10][11][12][13][14] as most of the off-the-shelf systems are monolingual and cannot handle code-switched speech. Our previous research has focused on developing an all-in-one CS ASR system using a Frisian-Dutch bilingual acoustic and language model that allows language switches [11,15].…”

Section: Introductionmentioning

confidence: 99%

Code-Switching Detection Using ASR-Generated Language Posteriors

Wang

Yılmaz

Derinel³

et al. 2019

Self Cite

Code-switching (CS) detection refers to the automatic detection of language switches in code-mixed utterances. This task can be achieved by using a CS automatic speech recognition (ASR) system that can handle such language switches. In our previous work, we have investigated the code-switching detection performance of the Frisian-Dutch CS ASR system by using the time alignment of the most likely hypothesis and found that this technique suffers from over-switching due to numerous very short spurious language switches. In this paper, we propose a novel method for CS detection aiming to remedy this shortcoming by using the language posteriors which are the sum of the framelevel posteriors of phones belonging to the same language. The CS ASR-generated language posteriors contain more complete language-specific information on frame level compared to the time alignment of the ASR output. Hence, it is expected to yield more accurate and robust CS detection. The CS detection experiments demonstrate that the proposed language posterior-based approach provides higher detection accuracy than the baseline system in terms of equal error rate. Moreover, a detailed CS detection error analysis reveals that using language posteriors reduces the false alarms and results in more robust CS detection.