Liaison and Pronunciation Learning in End-to-End Text-to-Speech in French

Abstract

Sequence-to-sequence (S2S) TTS models like Tacotron have grapheme-only inputs when trained fully end-to-end. Grapheme inputs map to phone sounds depending on context, which traditionally is handled by extensive preprocessing in the TTS front-end. However, French orthography does not provide a clear one-to-one mapping between graphemes and sounds, and in English, which similarly has rather non-phonetic orthography, pronunciations are a significant cause of error in S2S-TTS with grapheme-inputs. In this paper, we test implicit pronunciation knowledge where graphemes do not map directly to phones. Implicit pronunciation knowledge learnt in S2S-TTS is similar to a standalone grapheme-to-phoneme (G2P) model, which makes explicit phone predictions at the sequential level. We find grapheme-input S2S-TTS makes implicit pronunciation errors similar to explicit G2P models -notably for foreign names. In a traditional front-end pipeline, there are also postlexical rules which modify G2P output at the sequential level. In French, post-lexical rules require a deep knowledge of linguistic structure in a process called Liaison. Without explicit rules, we find S2S-TTS with grapheme-inputs over-inserts Liaison sounds, leading to a significant preference for a phonebased equivalent. By testing with linguistically-motivated stimuli, we observe differences that would otherwise go undetected.

Advocating for text input in multi-speaker text-to-speech systems

“…Note that letterto-sound (L2S) front-ends have poor performance on lexicons: [14] report 4.6% Phoneme Error Rate (PER) vs. 19.88% Word Error Rate (WER) on the CMUDict dataset using Token-Level Ensemble Distillation, while [15] report similar performance on CMUDict, Pronlex and NetTalk using encoder-decoder models. L2S front-ends for phonetic-to-speech synthesis are thus likely to produce similar PER as implicit L2S conversion reported by end-to-end TTS for English [1] or French [16].…”