Non-Native Text-to-Speech Preserving Speaker Individuality Based on Partial Correction of Prosodic and Phonetic Characteristics

Abstract

SUMMARYThis paper presents a novel non-native speech synthesis technique that preserves the individuality of a non-native speaker. Crosslingual speech synthesis based on voice conversion or Hidden Markov Model (HMM)-based speech synthesis is a technique to synthesize foreign language speech using a target speaker's natural speech uttered in his/her mother tongue. Although the technique holds promise to improve a wide variety of applications, it tends to cause degradation of target speaker's individuality in synthetic speech compared to intra-lingual speech synthesis. This paper proposes a new approach to speech synthesis that preserves speaker individuality by using non-native speech spoken by the target speaker. Although the use of non-native speech makes it possible to preserve the speaker individuality in the synthesized target speech, naturalness is significantly degraded as the synthesized speech waveform is directly affected by unnatural prosody and pronunciation often caused by differences in the linguistic systems of the source and target languages. To improve naturalness while preserving speaker individuality, we propose (1) a prosody correction method based on model adaptation, and (2) a phonetic correction method based on spectrum replacement for unvoiced consonants. The experimental results using English speech uttered by native Japanese speakers demonstrate that (1) the proposed methods are capable of significantly improving naturalness while preserving the speaker individuality in synthetic speech, and (2) the proposed methods also improve intelligibility as confirmed by a dictation test. key words: cross-lingual speech synthesis, English-Read-by-Japanese, speaker individuality, HMM-based speech synthesis, prosody correction, phonetic correction

Prosody Correction Preserving Speaker Individuality for Chinese-Accented Japanese HMM-Based Text-to-Speech Synthesis

“…On the other hand, delta feature correction brought no significant improvements and sometimes caused significant degradation of naturalness. Also, power correction, which was effective in Japanese-accented English [4], also brought no improvements. This is because power is not dominant in Chinese and Japanese speech.…”

“…Therefore, the stress and duration of Japanese-accented English speech are significantly different from those of native English speech. Correction of such features for Japanese-accented English text-to-speech synthesis can be done by partial adaptation of HSMMs [4]. First, a native English speaker's HSMMs are trained using the speaker's speech data.…”

“…Considering prosody system differences between Japanese and Chinese, we empirically investigate a prosody correction method that preserves speaker individuality. Furthermore, we also investigate the use of other correction methods that are not investigated by Oshima et al [4]. The experimental result demonstrates that duration and F 0 correction significantly improve the naturalness while preserving speaker individuality, regardless of the non-native speakers' level of Japanese proficiency.…”

“…To improve the naturalness of Japanese-accented English (i.e., English spoken by Japanese) text-to-speech synthesis, Oshima et al proposed prosody correction method that preserve speaker individuality [4]. Using frameworks of HMM-based text-to-speech synthesis and model adaptation [5], the HMM parameters are partially updated to fit the non-native speaker's speech parameters while fixing the remaining HMM parameters of the native speaker's speech.…”

NAUTILUS: A Versatile Voice Cloning System

“…In simple words, the goal of scenario B was to reproduce the accent of non-native speakers. This scenario is closely related to reducing accents [67], [68] or controlling accents [24] tasks.…”