Automatic Speech Recognition and Topic Identification from Speech for Almost-Zero-Resource Languages

Abstract

Automatic speech recognition (ASR) systems often need to be developed for extremely low-resource languages to serve enduses such as audio content categorization and search. While universal phone recognition is natural to consider when no transcribed speech is available to train an ASR system in a language, adapting universal phone models using very small amounts (minutes rather than hours) of transcribed speech also needs to be studied, particularly with state-of-the-art DNN-based acoustic models. The DARPA LORELEI program provides a framework for such very-low-resource ASR studies, and provides an extrinsic metric for evaluating ASR performance in a humanitarian assistance, disaster relief setting. This paper presents our Kaldi-based systems for the program, which employ a universal phone modeling approach to ASR, and describes recipes for very rapid adaptation of this universal ASR system. The results we obtain significantly outperform results obtained by many competing approaches on the NIST LoReHLT 2017 Evaluation datasets.

Pretraining by Backtranslation for End-to-End ASR in Low-Resource Settings

“…Appropriate pretraining of the encoder and decoder reduced the WER by 20% absolute in the 4h Italian set, to 56.2%. This performance has been shown to still be usable for some downstream tasks such as topic identification in low-resource settings [29].…”

Low-Resource Contextual Topic Identification on Speech

“…To solve the LORELEI task, prior work [8] used a mismatched ASR to directly decode IL speech, while [9] proposed sharing common phonemic representation among languages and transferring acoustic models trained on higher-resource (potentially related) language(s). After ASR, [8,9] translated both development (dev) and incident languages into English words, used the translated dev language data along with the given topic label annotations to learn English-language topic models and then classify the translated IL data.…”

“…To solve the LORELEI task, prior work [8] used a mismatched ASR to directly decode IL speech, while [9] proposed sharing common phonemic representation among languages and transferring acoustic models trained on higher-resource (potentially related) language(s). After ASR, [8,9] translated both development (dev) and incident languages into English words, used the translated dev language data along with the given topic label annotations to learn English-language topic models and then classify the translated IL data. Additionally, instead of using ASR to convert speech into sequences of words, [10,11,9] also investigated unsupervised techniques to automatically discover and decode IL speech segments into phone-like units via acoustic unit discovery (AUD), or into wordlike units via unsupervised term discovery (UTD).…”

“…We then transfer these models to a new language via a pronunciation lexicon with the same phonemic representation as used in training. We refer to this ASR as universal phone set ASR and we use the same approach as in [9]. Following [9], for experiments on Tigrinya and Oromo, we use the same selection of 10 BABEL languages for ASR training (∼600h).…”

“…We refer to this ASR as universal phone set ASR and we use the same approach as in [9]. Following [9], for experiments on Tigrinya and Oromo, we use the same selection of 10 BABEL languages for ASR training (∼600h). For Russian, we use 10h subsets of 21 BABEL languages (∼200h) in training (all except Haitian, Vietnamese, Amharic, Georgian).…”