Fast Language Adaptation Using Phonological Information

Abstract

Phoneme-based multilingual connectionist temporal classification (CTC) model is easily extensible to a new language by concatenating parameters of the new phonemes to the output layer. In the present paper, we improve cross-lingual adaptation in the context of phoneme-based CTC models by using phonological information. A universal (IPA) phoneme classifier is first trained on phonological features generated from a phonological attribute detector. When adapting the multilingual CTC to a new, never seen, language, phonological attributes of the unseen phonemes are derived based on phonology and fed into the phoneme classifier. Posteriors given by the classifier are used to initialize the parameters of the unseen phonemes when extending the multilingual CTC output layer to the target language. Adaptation experiments show that the proposed initialization approaches further improve the cross-lingual adaptation on CTC models and yield significant improvements over Deep Neural Network / Hidden Markov Model (DNN/HMM)-based adaptation using limited data.

A Self-Supervised Model for Language Identification Integrating Phonological Knowledge

“…[17] applied the articulatory features (AFs) to Deep Bottleneck (DBN) features based ivector and xvector systems, which can get better performance than baseline. Because the AFs are language-independent features, there are many researches focused on multilingual speech recognition [18][19][20]. Previous studies generally take a bottom-up approach and train phonological feature detectors, and here we jointly train the phonological labels and the acoustic frames reconstruction.…”

Multi-Spectral Widefield Microscopy of the Beating Heart Through Post-Acquisition Synchronization and Unmixing

“…In addition, we developed a new approach to initialize the model parameters by incorporating phonological information. It was demonstrated that the proposed approach results in better and faster convergence in cross-lingual adaptation [Tong et al, 2018b].…”

An Investigation of Multilingual ASR Using End-to-end LF-MMI

“…Moreover, a universal phoneme-based model is easily extensible to unseen phonemes when adapted to a new language [12]. With this motivation, and following our previous work [12,13], we propose a multilingual architecture that uses a universal output label set consisting of the union of all phonemes from the multiple languages. This universal phone set can be either derived in a data-driven way, or obtained from the International Phonetic Alphabet (IPA).…”

“…Multilingual ASR and cross-lingual adaptation can benefit more from these properties: language-specific prerequisite systems are no longer required; cross-lingual adaptation from an IPA-based system can be done simply by extending the output layer to new phonemes in a target language [12]. CTC training has been shown to be a promising alternative to the traditional DNN-HMM system for both multilingual ASR and cross-lingual adaptation [12,13].…”