Failure transitions for joint n-gram models and G2p conversion

Novak, Josef R.; Minematsu, Nobuaki; Hirose, Keikichi

doi:10.21437/interspeech.2013-449

Cited by 12 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traditionally G2P is done with joint-sequence n-gram models [1,2], [13,14]. While these are still considered to be state of the art models best designed for single lexicons, they require explicit alignment information to be provided during training.…”

Section: Related Workmentioning

confidence: 99%

Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion

2019

View full text Add to dashboard Cite

Grapheme-to-phoneme (G2P) models are a key component in Automatic Speech Recognition (ASR) systems, such as the ASR system in Alexa, as they are used to generate pronunciations for out-of-vocabulary words that do not exist in the pronunciation lexicons (mappings like "e c h o" → "E k oU ").Most G2P systems are monolingual and based on traditional joint-sequence based n-gram models [1,2]. As an alternative, we present a single end-to-end trained neural G2P model that shares same encoder and decoder across multiple languages. This allows the model to utilize a combination of universal symbol inventories of Latin-like alphabets and cross-linguistically shared feature representations. Such model is especially useful in the scenarios of low resource languages and code switching/foreign words, where the pronunciations in one language need to be adapted to other locales or accents. We further experiment with word language distribution vector as an additional training target in order to improve system performance by helping the model decouple pronunciations across a variety of languages in the parameter space. We show 7.2% average improvement in phoneme error rate over low resource languages and no degradation over high resource ones compared to monolingual baselines.

show abstract

Section: Related Workmentioning

confidence: 99%

Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion

2019

View full text Add to dashboard Cite

show abstract

“…We allow failure transitions [2,38] in the target WFA, which are taken only when no immediate match is possible at a given state, for compactness. For example, in the WFA representation of a backoff k-gram model, failure transitions can compactly implement the backoff [35,18,5,43,31]. The inclusion of failure transitions will complicate our analysis and algorithms but is highly desirable in applications such as keyboard decoding.…”

Section: Introductionmentioning

confidence: 99%

Approximating probabilistic models as weighted finite automata

Suresh¹,

Roark²,

Riley³

et al. 2019

Preprint

View full text Add to dashboard Cite

Weighted finite automata (WFA) are often used to represent probabilistic models, such as n-gram language models, since they are efficient for recognition tasks in time and space. The probabilistic source to be represented as a WFA, however, may come in many forms. Given a generic probabilistic model over sequences, we propose an algorithm to approximate it as a weighted finite automaton such that the Kullback-Leiber divergence between the source model and the WFA target model is minimized. The proposed algorithm involves a counting step and a difference of convex optimization step, both of which can be performed efficiently. We demonstrate the usefulness of our approach on various tasks, including distilling n-gram models from neural models, building compact language models, and building open-vocabulary character models. The algorithms used for these experiments are available in an open-source software library.

show abstract

Algorithms for Weighted Finite Automata with Failure Transitions

Allauzen

Riley

2018

Implementation and Application of Automata

View full text Add to dashboard Cite

Failure transitions for joint n-gram models and G2p conversion

Cited by 12 publications

References 6 publications

Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion

Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion

Approximating probabilistic models as weighted finite automata

Algorithms for Weighted Finite Automata with Failure Transitions

Contact Info

Product

Resources

About