Towards Zero-Shot Learning for Automatic Phonemic Transcription

Abstract

Automatic phonemic transcription tools are useful for low-resource language documentation. However, due to the lack of training sets, only a tiny fraction of languages have phonemic transcription tools. Fortunately, multilingual acoustic modeling provides a solution given limited audio training data. A more challenging problem is to build phonemic transcribers for languages with zero training data. The difficulty of this task is that phoneme inventories often differ between the training languages and the target language, making it infeasible to recognize unseen phonemes. In this work, we address this problem by adopting the idea of zero-shot learning. Our model is able to recognize unseen phonemes in the target language without any training data. In our model, we decompose phonemes into corresponding articulatory attributes such as vowel and consonant. Instead of predicting phonemes directly, we first predict distributions over articulatory attributes, and then compute phoneme distributions with a customized acoustic model. We evaluate our model by training it using 13 languages and testing it using 7 unseen languages. We find that it achieves 7.7% better phoneme error rate on average over a standard multilingual model.

Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview

“…More broadly there has been extensive work on domain adaptation and transfer learning in machine learning, reviewed by Kouw and Loog [46]. This includes work on few-shot learning [47]- [49] and normalizing flows [50], [51]. Normalizing flows which provide a probabilistic framework for feature transformations, were first developed for speech recognition as Gaussianization [52], and more recently have been applied to speech synthesis [53] and voice transformation [54].…”

Phone Distribution Estimation for Low Resource Languages

“…One critical issue with most multilingual recognition models is that their phone coverage is hardly complete [19]. For example, our trained model could cover around 200 phones, whereas the Phoible inventory has around 2000 distinct phones.…”

Simple and Effective Zero-shot Cross-lingual Phoneme Recognition

“…Zero-shot transfer learning addresses this by training a single multilingual model on the labeled data of several languages to enable zero-shot transcription of unseen languages [16,17,18,19,17,20]. Models usually have a common encoder that extracts acoustic information from speech audio and then predict either a shared phoneme vocabulary [17,16] or language-specific phonemes [1,20,21]. The former requires either phonological units that are agnostic to any particular language such as articulatory features [20] or global phones [22,17].…”