Discrete Duration Model for Speech Synthesis

Abstract

The acoustic model and the duration model are the two major components in statistical parametric speech synthesis (SPSS) systems. The neural network based acoustic model makes it possible to model phoneme duration at phone-level instead of state-level in conventional hidden Markov model (HMM) based SPSS systems. Since the duration of phonemes is countable value, the distribution of the phone-level duration is discrete given the linguistic features, which means the Gaussian hypothesis is no longer necessary. This paper provides an investigation on the performance of LSTM-RNN duration model that directly models the probability of the countable duration values given linguistic features using cross entropy as criteria. The multitask learning is also experimented at the same time, with a comparison to the standard LSTM-RNN duration model in objective and subjective measures. The result shows that directly modeling the discrete distribution has its benefit and multi-task model achieves better performance in phone-level duration modeling.

Duration Modeling with Global Phoneme-Duration Vectors

“…Technically, with advances in learning a continuous space vector representation of words in an unsupervised manner [10], large amounts of unannotated data can be used to improve TTS components [11][12] [13]. Moreover, a bidirectional recurrent neural network (BRNN) [14] [15] and bidirectional long short-term memory are well known to be powerful for sequence modeling [7] [16] [17]. This leads us to consider a basic approach for predicting duration using phonemes only but with advanced technology for sequence modeling and unsupervised learning.…”

“…Recently, the deep neural network technique has grown so fast that it has become the core in most data-driven systems, including TTS systems [3]. Neural network approaches have also been widely adopted to model duration [6] [7] [4]. However, most approaches [8][3] [6] [7] [4] predict phoneme duration using the full context labels that represent phonemes in context, including linguistic features, such as stress, and positional features, such as the relative positions of different segment levels (phoneme, syllable, and word) inside higher-level segments.…”

“…Neural network approaches have also been widely adopted to model duration [6] [7] [4]. However, most approaches [8][3] [6] [7] [4] predict phoneme duration using the full context labels that represent phonemes in context, including linguistic features, such as stress, and positional features, such as the relative positions of different segment levels (phoneme, syllable, and word) inside higher-level segments. A front-end tool is used to extract the contextual features from text and an embedding layer to represent the linguistic features along with duration model training [7].…”

“…However, most approaches [8][3] [6] [7] [4] predict phoneme duration using the full context labels that represent phonemes in context, including linguistic features, such as stress, and positional features, such as the relative positions of different segment levels (phoneme, syllable, and word) inside higher-level segments. A front-end tool is used to extract the contextual features from text and an embedding layer to represent the linguistic features along with duration model training [7]. By contrast, prosodic features represent paralinguistic information, such as the speaker's emotional state and speaking Space vector representations of phonemes and probabilistic mean duration…”

Duration modeling using DNN for Arabic speech synthesis

“…Furthermore, recurrent networks architecture, like long short-term memory (LSTM) and Bidirectional-LSTM (BLSTM) are powerful models in sequential modeling. Therefore, deep LSTM and BLSTM are investigated to model the relationship between linguistic features and phoneme duration [24]. The input features could be the same as those used in HTS, since they cover most of the phonological, linguistic and contextual data with the addition of Arabic specific features regarding vowel quantity and consonant gemination.…”