A high quality text-to-speech system composed of multiple neural networks

Abstract

While neural networks have been employed to handle several different text-to-speech tasks, ours is the first system to use neural networks throughout, for both linguistic and acoustic processing. We divide the text-to-speech task into three subtasks, a linguistic module mapping from text to a linguistic representation, an acoustic module mapping from the linguistic representation to speech, and a video module mapping from the linguistic representation to animated images. The linguistic module employs a letter-to-sound neural network and a postlexical neural network. The acoustic module employs a duration neural network and a phonetic neural network. The visual neural network is employed in parallel to the acoustic module to drive a talking head. The use of neural networks that can be retrained on the characteristics of different voices and languages affords our system a degree of adaptability and naturalness heretofore unavailable.

Improving Trajectory Modelling for DNN-Based Speech Synthesis by Using Stacked Bottleneck Features and Minimum Generation Error Training

“…More recently, following on from successes in automatic speech recognition [9], artificial neural networks have reemerged as acoustic models for SPSS [10]. By the 1990s, artificial neural networks had already been employed as feature extractors from text input to produce linguistic features [11], as acoustic models to map linguistic features to vocoder parameters [12], [13], [14], and to predict segment durations [15]. One prominent theme in more recent studies is the use of neural architectures to replace Gaussian mixture models (GMMs) associated with leaf nodes of decision trees, such as the restricted Boltzmann machines (RBMs) in [16], where RBMs were claimed to better learn spectral detail, resulting in better quality synthesised speech.…”

TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction

“…A typical statistical parametric TTS pipeline has the following stages: grapheme-to-phoneme conversion, a phoneme duration predictor, an acoustic frame-level feature generator, and a vocoder [11]. Neural networks have been used for TTS in 1990 [12,13,14,15]. The deep learning re-introduced NNs to TTS: Zen et al [16,17] proposed a hybrid NN-parametric TTS model, where deep NNs are used to predict the phoneme duration and to generate frame-level acoustic features.…”

Implementation of Three Text to Speech Systems for Kurdish Language

“…As a result, for extracting allophones we used a neural network. Because their learning power, neural networks can learn from a database and can recognize allophones properly [35]. Fig.…”