Language diarization for semi-supervised bilingual acoustic model training

Yılmaz, Emre; McLaren, Mitchell; Heuvel, H. van den; Leeuwen, David A. van

doi:10.1109/asru.2017.8268921

Cited by 21 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The acoustic data augmentation relies on available monolingual acoustic resources from the high-resourced mixed language (Dutch). Using more monolingual Dutch speech for acoustic model training has found to be effective in improving the general ASR performance, only after increasing the in-domain CS data applying the semi-supervised techniques described in [17,21].…”

Section: Introductionmentioning

confidence: 99%

Multi-Graph Decoding for Code-Switching ASR

et al. 2019

View full text Add to dashboard Cite

In the FAME! Project, a code-switching (CS) automatic speech recognition (ASR) system for Frisian-Dutch speech is developed that can accurately transcribe the local broadcaster's bilingual archives with CS speech. This archive contains recordings with monolingual Frisian and Dutch speech segments as well as Frisian-Dutch CS speech, hence the recognition performance on monolingual segments is also vital for accurate transcriptions. In this work, we propose a multi-graph decoding and rescoring strategy using bilingual and monolingual graphs together with a unified acoustic model for CS ASR. The proposed decoding scheme gives the freedom to design and employ alternative search spaces for each (monolingual or bilingual) recognition task and enables the effective use of monolingual resources of the high-resourced mixed language in low-resourced CS scenarios. In our scenario, Dutch is the high-resourced and Frisian is the low-resourced language. We therefore use additional monolingual Dutch text resources to improve the Dutch language model (LM) and compare the performance of single-and multi-graph CS ASR systems on Dutch segments using larger Dutch LMs. The ASR results show that the proposed approach outperforms baseline single-graph CS ASR systems, providing better performance on the monolingual Dutch segments without any accuracy loss on monolingual Frisian and code-mixed segments.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Graph Decoding for Code-Switching ASR

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Then, we extract word-level segmentation files for each LM weight. By comparing these alignments with the ground truth word-level alignments, a time-based CS detection accuracy metric is calculated [19]. CS detection accuracy is evaluated by reporting the equal error rates (EER) calculated based on the detection error tradeoff (DET) graph [39] plotted for visualizing the CS detection performance.…”

Section: Recognition and Cs Detection Experimentsmentioning

confidence: 99%

Code-Switching Detection with Data-Augmented Acoustic and Language Models

Yılmaz

Heuvel

Leeuwen

2018

6th Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU 2018)

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate the code-switching detection performance of a code-switching (CS) automatic speech recognition (ASR) system with data-augmented acoustic and language models. We focus on the recognition of Frisian-Dutch radio broadcasts where one of the mixed languages, namely Frisian, is under-resourced. Recently, we have explored how the acoustic modeling (AM) can benefit from monolingual speech data belonging to the high-resourced mixed language. For this purpose, we have trained state-of-the-art AMs on a significantly increased amount of CS speech by applying automatic transcription and monolingual Dutch speech. Moreover, we have improved the language model (LM) by creating CS text in various ways including text generation using recurrent LMs trained on existing CS text. Motivated by the significantly improved CS ASR performance, we delve into the CS detection performance of the same ASR system in this work by reporting CS detection accuracies together with a detailed detection error analysis.

show abstract

“…Code-switching can be broadly divided into two groups [5]: inter-sentential switching -the alternation is between sentences (also called extrasentential), and intra-sentential switching -the alternation is within sentences (it can also include intra-word). With the rapidly growing of bilingual/multilingual population, CS is no longer a phenomenon relevant for minority languages, which are affected by majority languages, but it also concerns majority languages influenced by lingua francas, such as English and French, as properly pointed out in [6]. Despite the recent significant advances witnessed in the field of automatic speech recognition (ASR) [7], ASR systems have unfortunately still limited capability in tackling the code-switching problem, especially intra-sentential switching.…”

Section: Introductionmentioning

confidence: 99%

Exploring Retraining-free Speech Recognition for Intra-sentential Code-switching

Huang

Zhuang

Liu

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we present our initial efforts for building a codeswitching (CS) speech recognition system leveraging existing acoustic models (AMs) and language models (LMs), i.e., no training required, and specifically targeting intra-sentential switching. To achieve such an ambitious goal, new mechanisms for foreign pronunciation generation and language model (LM) enrichment have been devised. Specifically, we have designed an automatic approach to obtain high quality pronunciation of foreign language (FL) words in the native language (NL) phoneme set using existing acoustic phone decoders and an LSTM-based grapheme-to-phoneme (G2P) model. Improved accented pronunciations have thus been obtained by learning foreign pronunciations directly from data. Furthermore, a codeswitching LM was deployed by converting the original NL LM into a CS LM using translated word pairs and borrowing statistics for the NL LM. Experimental evidence clearly demonstrates that our approach better deals with accented foreign pronunciations than techniques based on human labeling. Moreover, our best system achieves a 55.5% relative word error rate reduction from 34.4%, obtained with a conventional monolingual ASR system, to 15.3% on an intra-sentential CS task without harming the monolingual recognition accuracy.

show abstract

Language diarization for semi-supervised bilingual acoustic model training

Cited by 21 publications

References 26 publications

Multi-Graph Decoding for Code-Switching ASR

Multi-Graph Decoding for Code-Switching ASR

Code-Switching Detection with Data-Augmented Acoustic and Language Models

Exploring Retraining-free Speech Recognition for Intra-sentential Code-switching

Contact Info

Product

Resources

About