Context-dependent acoustic modeling based on hidden maximum entropy model for statistical parametric speech synthesis

Abstract

Decision tree-clustered context-dependent hidden semi-Markov models (HSMMs) are typically used in statistical parametric speech synthesis to represent probability densities of acoustic features given contextual factors. This paper addresses three major limitations of this decision tree-based structure: (i) The decision tree structure lacks adequate context generalization. (ii) It is unable to express complex context dependencies. (iii) Parameters generated from this structure represent sudden transitions between adjacent states. In order to alleviate the above limitations, many former papers applied multiple decision trees with an additive assumption over those trees. Similarly, the current study uses multiple decision trees as well, but instead of the additive assumption, it is proposed to train the smoothest distribution by maximizing entropy measure. Obviously, increasing the smoothness of the distribution improves the context generalization. The proposed model, named hidden maximum entropy model (HMEM), estimates a distribution that maximizes entropy subject to multiple moment-based constraints. Due to the simultaneous use of multiple decision trees and maximum entropy measure, the three aforementioned issues are considerably alleviated. Relying on HMEM, a novel speech synthesis system has been developed with maximum likelihood (ML) parameter re-estimation as well as maximum output probability parameter generation. Additionally, an effective and fast algorithm that builds multiple decision trees in parallel is devised. Two sets of experiments have been conducted to evaluate the performance of the proposed system. In the first set of experiments, HMEM with some heuristic context clusters is implemented. This system outperformed the decision tree structure in small training databases (i.e., 50, 100, and 200 sentences). In the second set of experiments, the HMEM performance with four parallel decision trees is investigated using both subjective and objective tests. All evaluation results of the second experiment confirm significant improvement of the proposed system over the conventional HSMM.

Convolutional Neural Network Based Speaker De-Identification

“…STRAIGHT extracts 513-D spectral envelope (SP), 513-D aperiodicity (AP) features as well as 1-D fundamental frequency (F0). We employ speech signal processing toolkit (SPTK) to convert SP to 40-D Mel-cepstral coefficients (MCEP) [15]. The de-identification is only applied to MCEP and F0 features; the AP features are directly mapped from the source speaker to the de-identified speaker.…”

A tutorial review on entropy-based handcrafted feature extraction for information fusion

“…In paper [60] , authors successfully use neural-scaled entropy (NSE) and cochlearscaled entropy (CSE) in order to predict the effects of nonlinear frequency compression on speech perception due to sensorineural hearing loss (SNHL). In speech synthesis, the major limitations imposed with the use of tree-clustered context-dependent hidden semi-Markov models adopted are fruitfully circumvented by the authors of paper [40] based on entropy.…”

Soft context clustering for F0 modeling in HMM-based speech synthesis

“…F0 modeling with additive structures has also been used to express the relationship between contextual factors and the F0 trajectory [44][45][46][47][48][49][50][51][52][53][54]. Contextual additive modeling [45][46][47][48] assumes model parameters to be a sum of multiple independent components, each having different context dependencies; therefore, different decision trees have to be trained for them.…”