Bilingual acoustic modeling with state mapping and three-stage adaptation for transcribing unbalanced code-mixed lectures

Abstract

This paper presents a bilingual acoustic modeling approach for transcribing Mandarin-English code-mixed lectures with highly unbalanced language distribution. Special terminologies for the content were produced in the guest language of English (about 15%) and embedded in the utterances produced in the host lan guage of Mandarin (about 85%). The code-mixing nature of the target corpus and the very small percentage of the English data made the task difficult. State mapping and merging approaches plus three stages of model adaptation handles the above problem. Significant improvements in recognition accuracy were obtained in the experiment with a real bilingual code-mixed lecture corpus recorded at National Taiwan University. The code-mixing situation considered is actually very natural in the spoken language of the daily lives of many people in the globalized world today.

Jee haan, I’d like both, por favor: Elicitation of a Code-Switched Corpus of Hindi–English and Spanish–English Human–Machine Dialog

“…There is comparatively less work in the literature on automated analysis of code-switched speech, partially due to the relative lack of structured corpora (as compared to those for textbased work) and also potentially because it also poses yet another significant challenge in the form of speech recognition for multiple languages. Nonetheless, some researchers have made strong strides in spoken corpus development to support such research in certain language pairs, for instance, Mandarin-English [21,22], Cantonese-English [23] and Hindi-English [24], which have in turn led to developments in automatic speech recognition [25,26] and language modeling [27]. However, these are limited; there remains a need for more codeswitched speech resources in these and other languages to spur research into the automated processing and analysis of such data.…”

Minimum Phone Error model training on merged acoustic units for transcribing bilingual code-switched speech

“…Usually the similarity and differences between the phonemes and the unique characteristics for some phonemes are difficult to measure quantitatively. Many approaches have been proposed to merge acoustic units on different levels to handle these problems [3] [4][5] [6] [7]. In general, bilingual speech can be classified into two categories.…”