Zero-shot Learning for Grapheme to Phoneme Conversion with Language Ensemble

Li, Xinjian; Metze, Florian; Mortensen, David A.; Watanabe, Soichi; Black, Alan W.

doi:10.18653/v1/2022.findings-acl.166

Cited by 6 publications

(10 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other parameters follow the original literature [24]. For the pronunciation model, we use the multilingual model proposed in the previous literature and its implementation [30] 1 . For the language model, we first download the complete dataset from Crúbadán's website [33], which results in 1909 languages after cleaning.…”

Section: Methodsmentioning

confidence: 99%

“…However, the majority of the languages do not have any accessible dictionaries or rules, therefore we consider an approximated pronunciation model δpm instead. In particular, we apply a recently proposed multilingual G2P model as our pronunciation model [30]. For any target language ltarget, this G2P model selects top-k nearest languages: ltopk ∈ KNN(ltarget) whose training set is available, then during the inference, it first propose k hypothesis using each nearest language model δ l topk , the models are ensembled by combining hypothesis into a lattice to emit the most-likely approximated sequence:…”

Section: Pronunciation Modelmentioning

confidence: 99%

See 1 more Smart Citation

ASR2K: Speech Recognition for Around 2000 Languages without Audio

Li¹,

Metze²,

Mortensen³

et al. 2022

Interspeech 2022

Self Cite

View full text Add to dashboard Cite

Most recent speech recognition models rely on large supervised datasets, which are unavailable for many low-resource languages. In this work, we present a speech recognition pipeline that does not require any audio for the target language. The only assumption is that we have access to raw text datasets or a set of n-gram statistics. Our speech pipeline consists of three components: acoustic, pronunciation, and language models. Unlike the standard pipeline, our acoustic and pronunciation models use multilingual models without any supervision. The language model is built using n-gram statistics or the raw text dataset. We build speech recognition for 1909 languages by combining it with Crúbadán: a large endangered languages n-gram database. Furthermore, we test our approach on 129 languages across two datasets: Common Voice and CMU Wilderness dataset. We achieve 50% CER and 74% WER on the Wilderness dataset with Crúbadán statistics only and improve them to 45% CER and 69% WER when using 10000 raw text utterances.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Pronunciation Modelmentioning

confidence: 99%

ASR2K: Speech Recognition for Around 2000 Languages without Audio

Li¹,

Metze²,

Mortensen³

et al. 2022

Interspeech 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this study, we select the similar languages for the cross-lingual transfer based on language family defined in Glottolog [43]. Language selection based on language family is used in previous studies [44], [4] for spoken language processing tasks. Specifically, as mentioned in § IV-A, we choose French and Spanish from the same Romance languages for adaptation to Italian, while we use Malayalam and Telugu from the same Dravidian languages for adaptation to Tamil.…”

Section: B Text-based Adaptationmentioning

confidence: 99%

Text-Inductive Graphone-Based Language Adaptation for Low-Resource Speech Synthesis

Saeki,

Maiti,

et al. 2024

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Neural text-to-speech (TTS) systems have made significant progress in generating natural synthetic speech. However, neural TTS requires large amounts of paired training data, which limits its applicability to a small number of resource-rich languages. Previous work on low-resource TTS has addressed the data hungriness based on transfer learning from a multilingual model to low-resource languages, but it still relies heavily on the availability of paired data for the target languages. In this paper, we propose a text-inductive language adaptation framework for low-resource TTS to address the cost of collecting the paired data for low-resource languages. To inject textual knowledge during transfer learning, our framework employs a two-stage adaptation scheme that utilizes both text-only and supervised data for the target language. In the text-based adaptation stage, we update the language-aware embedding layer with a masked language model objective using text-only data for the target language. In the supervised adaptation stage, the entire TTS model is updated using paired data for the target language. We also propose a graphone-based multilingual training method that jointly uses graphemes and International Phonetic Alphabet symbols (referred to as graphones) for resource-rich languages, while using only graphemes for low-resource languages. This approach facilitates the transfer of pronunciation knowledge from resource-rich to low-resource languages. Through extensive evaluations, we demonstrate that 1) our framework with textbased adaptation outperforms the previous supervised transfer learning approach, 2) the proposed graphone-based training method further improves the performance of both multilingual TTS and low-resource language adaptation. With only 5 minutes of paired data for fine-tuning, our method achieved highly intelligible synthetic speech with the character error rates of around 6 % for a target language.

show abstract

“…Recently deep learning architectures for g2p developed based on recurrent neural networks [33], convolutional neural networks [34] and transformers [35]. Zero shot g2p learning techniques without explicit training data have been proposed, but they are based on the assumption that similar language families use the same orthography, which is not always true [36]. Phonetisaurus [37] is a data driven tool that learns the mapping rules statistically (joint sequence models) from a training dataset and builds weighted FSTs for g2p conversion.…”

Section: Related Workmentioning

confidence: 99%

“…For languages with regular grapheme to phoneme conversion patterns, knowledge-based g2p has been reported to produce good results [36], [38]. A set of sequential rewrite rules can be used to achieve this.…”

Section: Related Workmentioning

confidence: 99%

Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers

2022

View full text Add to dashboard Cite

In this article we present the design and the development of a knowledge based computational linguistic tool, Mlphon [em.el.foːɳ] for Malayalam language. Mlphon computationally models linguistic rules using finite state transducers and performs multiple functions including grapheme to phoneme (g2p) and phoneme to grapheme (p2g) conversions, syllabification, phonetic feature analysis and script grammar check. This open source software tool, released under MIT license, is developed as a one-stop solution to handle different speech related text processing tasks for automatic speech recognition, text to speech synthesis and nonspeech natural language processing tasks including syllable subword based language modeling, phoneme diversity analysis and text sanity check. The tool is evaluated on a manually crafted gold standard lexicon. Mlphon performs orthographic syllabification with 99% accuracy with a syllable error rate of 0.62% on the gold standard lexicon. For grapheme to phoneme conversion task, overall phoneme recognition accuracy of 99% with a phoneme error rate of 0.55% is obtained on gold standard lexicon. Additionally an extrinsic evaluation of Mlphon is performed by employing the pronunciation lexicon created using Mlphon, in Malayalam automatic speech recognition (ASR) task. Performance analysis in terms of the computation time of lexicon creation process and the word error rate (WER) on ASR task are presented along with a comparison over other automated tools for lexicon creation. Pronunciation lexicons with more than 100k commonly used Malayalam words in phonemised and syllabified forms is created and they are published as open language resources along with this work. We also demonstrate the usage of Mlphon on different natural language processing applications -syllable subword ASR, assisted pronunciation learning, phoneme diversity analysis and text sanity check. Being a knowledge based solution with open source code, Mlphon can be adapted to other languages of similar script nature.

show abstract

Zero-shot Learning for Grapheme to Phoneme Conversion with Language Ensemble

Cited by 6 publications

References 17 publications

ASR2K: Speech Recognition for Around 2000 Languages without Audio

ASR2K: Speech Recognition for Around 2000 Languages without Audio

Text-Inductive Graphone-Based Language Adaptation for Low-Resource Speech Synthesis

Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers

Contact Info

Product

Resources

About