From HMMS to DNNS: Where do the improvements come from?

Abstract

Deep neural networks (DNNs) have recently been the focus of much text-to-speech research as a replacement for decision trees and hidden Markov models (HMMs) in statistical parametric synthesis systems. Performance improvements have been reported; however, the configuration of systems evaluated makes it impossible to judge how much of the improvement is due to the new machine learning methods, and how much is due to other novel aspects of the systems. Specifically, whereas the decision trees in HMM-based systems typically operate at the state-level, and separate trees are used to handle separate acoustic streams, most DNN-based systems are trained to make predictions simultaneously for all streams at the level of the acoustic frame. This paper isolates the influence of three factors (machine learning method; state vs. frame predictions; separate vs. combined stream predictions) by building a continuum of systems along which only a single factor is varied at a time. We find that replacing decision trees with DNNs and moving from state-level to frame-level predictions both significantly improve listeners' naturalness ratings of synthetic speech produced by the systems. No improvement is found to result from switching from separate-stream to combined-stream predictions.

Adapting and controlling DNN-based speech synthesis using input codes

“…It has been reported that DNN-based techniques have improved the quality of synthetic speech significantly; cf. [10]. A new DNN baseline system [11] added to the Blizzard Challenge 2016 1 also turned out to be significantly better than the standard HMM-based baseline (that uses a toolkit called HTS [12]), again confirming the speech quality improvements brought on by deep learning approaches.…”

Improving Trajectory Modelling for DNN-Based Speech Synthesis by Using Stacked Bottleneck Features and Minimum Generation Error Training

“…The most popular way to use neural networks in SPSS is with a deep feed-forward neural network (DNN) as a conditional model to map linguistic features to vocoder parameters directly [22], [23], [24], [25], [26]. This can be viewed as replacing the decision tree used in HMM-based speech synthesis with a more powerful regression model [22], [27].…”

Adaptation and Frontend Features to Improve Naturalness in Found-Data Synthesis

“…Neural network based synthesis has recently produced very high-quality voices, and addresses some of the naturalness issues common to HMM-based voices. [17] found that the acrossclass averaging resulting from decision tree based context clustering is a major detractor of naturalness in HMM voice quality, and [18] found that replacing the decision trees with DNNs and the production of frame-level rather than state-level predictions substantially improved naturalness as well. Furthermore, [19] found that an HMM system trained on 100 hours of data was comparable in f0 correlation (an objective measure of naturalness) to a DNN system using only 10 hours.…”