Grapheme-to-phoneme conversion using Long Short-Term Memory recurrent neural networks

Abstract

Grapheme-to-phoneme (G2P) models are key components in speech recognition and text-to-speech systems as they describe how words are pronounced. We propose a G2P model based on a Long Short-Term Memory (LSTM) recurrent neural network (RNN). In contrast to traditional joint-sequence based G2P approaches, LSTMs have the flexibility of taking into consideration the full context of graphemes and transform the problem from a series of grapheme-to-phoneme conversions to a word-to-pronunciation conversion. Training joint-sequence based G2P require explicit graphemeto-phoneme alignments which are not straightforward since graphemes and phonemes don't correspond one-to-one. The LSTM based approach forgoes the need for such explicit alignments.We experiment with unidirectional LSTM (ULSTM) with different kinds of output delays and deep bidirectional LSTM (DBLSTM) with a connectionist temporal classification (CTC) layer. The DBLSTM-CTC model achieves a word error rate (WER) of 25.8% on the public CMU dataset for US English. Combining the DBLSTM-CTC model with a joint n-gram model results in a WER of 21.3%, which is a 9% relative improvement compared to the previous best WER of 23.4% from a hybrid system.

A knowledge-based pronunciation generation system for French

“…[campINg] X-SAMPA, extended speech assessment methods phonetic alphabet; IPA, international phonetic alphabet (Yvon et al, 1998;Béchet, 2001;de Mareüil et al, 2005;Rao et al, 2015). (Byrd & Tzoukermann, 1988;Gruaz et al, 1996).…”

Fix it where it fails: Pronunciation learning by mining error corrections from speech logs

“…There is a lot of research on applying machine learning for grapheme to phoneme conversion (G2P), including decision tree classifier to learn pronunciation rules [3], joint ngram model [4], maximum entroy model [5], active learning [6], and most recently recurrent neural network [7]. In this paper, instead of focuing on improving machine learning G2P techniques, we strive to learn pronunciations from recognition corrections data.…”

Do we need bigram alignment models? On the effect of alignment quality on transduction accuracy in G2P

“…The classical learning paradigm in each of these settings is to train a model on pairs of strings {(x, y)} and then to evaluate model performance on test data. While there are exceptions (e.g., (Rao et al, 2015)), most state-of-the-art modelings (e.g., (Jiampojamarn et al, 2007;Bisani and Ney, 2008;Jiampojamarn et al, 2008;Novak et al, 2012)) view string transduction as a two-stage process in which string pairs (x, y) in the training data are first aligned, and then a subsequent (e.g., sequence labeling) module is learned on the aligned data. ph oe n i x f i n I ks Table 1: Sample monotone many-to-many alignment between x = phoenix and y = finIks.…”