Testing the consistency assumption: Pronunciation variant forced alignment in read and spontaneous speech synthesis

Abstract

Forced alignment for speech synthesis traditionally aligns a phoneme sequence predetermined by the front-end text processing system. This sequence is not altered during alignment, i.e., it is forced, despite possibly being faulty. The consistency assumption is the assumption that these mistakes do not degrade models, as long as the mistakes are consistent across training and synthesis. We present evidence that in the alignment of both standard read prompts and spontaneous speech this phoneme sequence is often wrong, and that this is likely to have a negative impact on acoustic models. A latticebased forced alignment system allowing for pronunciation variation is implemented, resulting in improved phoneme identity accuracy for both types of speech. A perceptual evaluation of HMM-based voices showed that spontaneous models trained on this improved alignment also improved standard synthesis, despite breaking the consistency assumption.

Can We Generate Emotional Pronunciations for Expressive Speech Synthesis?

“…Rule-based pronunciation lattices have also been used to reduce inconsistencies between the phoneme sequence generated by the front-end text processing system (i.e. the phonetizer) and the phoneme sequence as transcribed in the training speech corpus [27]. However, building and maintaining such hand-crafted pronunciation lexicons designed by linguistic experts and transpose them to diverse speaking styles and expressivity is expensive and time consuming.…”

Synthesising Filled Pauses: Representation and Datamixing

“…The same result persisted even after a pronunciation variant forced alignment method was applied to compensate for the additional reductions and deletions present in the spontaneous speech. However, the test sentences used in [19] did not contain FPs, and other researchers have found that including these in voices based on spontaneous speech can improve their perceived naturalness to be on par with, or even better than, voices trained on read speech [20,16]. A test was thus performed to confirm whether this is also the case for the corpora used for the experiments reported in this paper.…”

“…The read and spontaneous speech corpora used in these experiments were the same as those described in [19]. The read corpus consisted of studio recordings of a female British English speaker (recorded at 96khz, 32 bit, downsampled to 48khz, 16 bit), the prompts used were the Arctic sentences [18] and the R. Dall, M. Tomalin, M. Wester corpus contained a total of 1125 sentences (66 mins: 20 mins silence, 46 mins speech).…”

“…SiRe [27] was used as the front-end and was modified to convert all vowels in FPs into each of four phones -/V/, /@/, /UHV/ and /UH@/. /UHV/ used the phone features of /V/, and /UH@/ the features of /@/.…”

“…In [13] it was shown that studio recordings of spontaneously produced speech are considered more natural than these standard "read" prompts, particularly when acoustic variation is considered. However, in [19] the overall naturalness of TTS output produced by a system trained on a spontaneous speech corpus never matched that of the voice trained on read speech. The same result persisted even after a pronunciation variant forced alignment method was applied to compensate for the additional reductions and deletions present in the spontaneous speech.…”