Deep neural networks employing Multi-Task Learning and stacked bottleneck features for speech synthesis

Wu, Zhizheng; Valentini-Botinhao, Cassia; Watts, Oliver; King, Simon

doi:10.1109/icassp.2015.7178814

Cited by 225 publications

(164 citation statements)

References 19 publications

Supporting

Mentioning

161

Contrasting

Unclassified

Order By: Relevance

“…The use of bottleneck features for modelling these dependencies seems reasonable. Bottleneck features have been widely used in speech recognition [15,16] and text-to-speech systems [17]. They can be used in a similar way for speech-driven head motion synthesis.…”

Section: Bottleneck Featuresmentioning

confidence: 99%

Bidirectional LSTM Networks Employing Stacked Bottleneck Features for Expressive Speech-Driven Head Motion Synthesis

Haag

Shimodaira

2016

Intelligent Virtual Agents

View full text Add to dashboard Cite

Abstract. Previous work in speech-driven head motion synthesis is centred around Hidden Markov Model (HMM) based methods and data that does not show a large variability of expressiveness in both speech and motion. When using expressive data, these systems often fail to produce satisfactory results. Recent studies have shown that using deep neural networks (DNNs) results in a better synthesis of head motion, in particular when employing bidirectional long short-term memory (BLSTM). We present a novel approach which makes use of DNNs with stacked bottleneck features combined with a BLSTM architecture to model context and expressive variability. Our proposed DNN architecture outperforms conventional feed-forward DNNs and simple BLSTM networks in an objective evaluation. Results from a subjective evaluation show a significant improvement of the bottleneck architecture over feed-forward DNNs.

show abstract

Section: Bottleneck Featuresmentioning

confidence: 99%

Bidirectional LSTM Networks Employing Stacked Bottleneck Features for Expressive Speech-Driven Head Motion Synthesis

Haag

Shimodaira

2016

Intelligent Virtual Agents

View full text Add to dashboard Cite

show abstract

“…Therefore, the SGD training can be done with the weighted sum of the task-specific objectives for each training sample, and the language model objective can be thought of as a regularization term. Similar settings of multitask learning for neural network models are employed in phoneme recognition for speech (Seltzer and Droppo, 2013) and speech synthesis (Wu et al, 2015) as well, but both of them use equal weights for all tasks.…”

Section: Related Workmentioning

confidence: 99%

Open-Domain Name Error Detection using a Multi-Task RNN

Cheng¹,

Fang²,

Ostendorf³

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Out-of-vocabulary name errors in speech recognition create significant problems for downstream language processing, but the fact that they are rare poses challenges for automatic detection, particularly in an open-domain scenario. To address this problem, a multi-task recurrent neural network language model for sentence-level name detection is proposed for use in combination with out-of-vocabulary word detection. The sentence-level model is also effective for leveraging external text data. Experiments show a 26% improvement in name-error detection F-score over a system using n-gram lexical features.

show abstract

“…The database consists of 2400 utterances for training, 70 for testing, recorded with a sample rate of 16kHz. The input features consist of 160 bottleneck features [23] as a compact, learned linguistic representation. For spectral features, either i) 50 regularized discrete cepstra (RDC) extracted from the amplitudes of the harmonic dynamic model (HDM) [24] or ii) 50 highly correlated log amplitudes from the perceptual dynamic sinusoidal model (PDM) [25] are used as real-valued spectral output.…”

Section: System Configurationmentioning

confidence: 99%

Initial investigation of speech synthesis based on complex-valued neural networks

Yamagishi

Richmond

et al. 2016

2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Although frequency analysis often leads us to a speech signal in the complex domain, the acoustic models we frequently use are designed for real-valued data. Phase is usually ignored or modelled separately from spectral amplitude. Here, we propose a complex-valued neural network (CVNN) for directly modelling the results of the frequency analysis in the complex domain (such as the complex amplitude). We also introduce a phase encoding technique to map real-valued data (e.g. cepstra or log amplitudes) into the complex domain so we can use the same CVNN processing seamlessly. In this paper, a fully complex-valued neural network, namely a neural network where all of the weight matrices, activation functions and learning algorithms are in the complex domain, is applied for speech synthesis. Results show its ability to model both complex-valued and real-valued data.

show abstract

Deep neural networks employing Multi-Task Learning and stacked bottleneck features for speech synthesis

Cited by 225 publications

References 19 publications

Bidirectional LSTM Networks Employing Stacked Bottleneck Features for Expressive Speech-Driven Head Motion Synthesis

Bidirectional LSTM Networks Employing Stacked Bottleneck Features for Expressive Speech-Driven Head Motion Synthesis

Open-Domain Name Error Detection using a Multi-Task RNN

Initial investigation of speech synthesis based on complex-valued neural networks

Contact Info

Product

Resources

About