Postfilters to Modify the Modulation Spectrum for Statistical Parametric Speech Synthesis

Takamichi, Shinnosuke; Toda, Tomoki; Black, Alan W.; Neubig, Graham; Sakti, Sakriani; Nakamura, Satoshi

doi:10.1109/taslp.2016.2522655

Cited by 62 publications

(39 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In particular, the common issue of oversmoothing is typically not reflected in these metrics. A recently proposed metric, the Modulation Spectrum (MS) [40], allows visualizing the spectral content of predicted time sequences. For instance, showing oversmoothing as a rolloff of higher modulation frequencies.…”

Section: • Modulation Spectrum (Ms) For Mel-generalized Coefficients mentioning

confidence: 99%

A Neural Parametric Singing Synthesizer Modeling Timbre and Expression from Natural Songs

2017

View full text Add to dashboard Cite

Abstract:We recently presented a new model for singing synthesis based on a modified version of the WaveNet architecture. Instead of modeling raw waveform, we model features produced by a parametric vocoder that separates the influence of pitch and timbre. This allows conveniently modifying pitch to match any target melody, facilitates training on more modest dataset sizes, and significantly reduces training and generation times. Nonetheless, compared to modeling waveform directly, ways of effectively handling higher-dimensional outputs, multiple feature streams and regularization become more important with our approach. In this work, we extend our proposed system to include additional components for predicting F0 and phonetic timings from a musical score with lyrics. These expression-related features are learned together with timbrical features from a single set of natural songs. We compare our method to existing statistical parametric, concatenative, and neural network-based approaches using quantitative metrics as well as listening tests.

show abstract

Section: • Modulation Spectrum (Ms) For Mel-generalized Coefficients mentioning

confidence: 99%

A Neural Parametric Singing Synthesizer Modeling Timbre and Expression from Natural Songs

2017

View full text Add to dashboard Cite

show abstract

“…However, this optimization framework often causes excessively smoothed speech parameters, making the converted speech sound muffled. To address this oversmoothing problem, there are several methods have been proposed, e.g., 1) a method to model additional features to sensitively capture the oversmoothing effect, such as global variance (GV) [27] and modulation spectrum (MS) [36], 2) a method to keep characteristics of natural speech parameters by partially using the source speech parameters, such as dynamic frequency warping (DFW) [37], and 3) a method to alleviate the averaging process to implement a sparse constraint as in the exemplar-based conversion [33,34].…”

Section: Conversion Functionmentioning

confidence: 99%

The Voice Conversion Challenge 2016

et al. 2016

Self Cite

View full text Add to dashboard Cite

This paper describes the Voice Conversion Challenge 2016 devised by the authors to better understand different voice conversion (VC) techniques by comparing their performance on a common dataset. The task of the challenge was speaker conversion, i.e., to transform the voice identity of a source speaker into that of a target speaker while preserving the linguistic content. Using a common dataset consisting of 162 utterances for training and 54 utterances for evaluation from each of 5 source and 5 target speakers, 17 groups working in VC around the world developed their own VC systems for every combination of the source and target speakers, i.e., 25 systems in total, and generated voice samples converted by the developed systems. These samples were evaluated in terms of target speaker similarity and naturalness by 200 listeners in a controlled environment. This paper summarizes the design of the challenge, its result, and a future plan to share views about unsolved problems and challenges faced by the current VC techniques.

show abstract

“…The over-smoothing effect is an issue in not only VC but also other speech synthesis techniques, such as text-to-speech synthesis. Hence, several approaches have been devised to reproduce the characteristics of natural speech [2], [6], [7]. On the other hand, VC can utilize not only those approaches, but also input speech information since the input and output parameters are often in the same domain (e.g., cepstrum).…”

Section: Introductionmentioning

confidence: 99%

Voice Conversion Using Input-to-Output Highway Networks

Saito

Takamichi

Saruwatari

2017

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

SUMMARYThis paper proposes Deep Neural Network (DNN)-based Voice Conversion (VC) using input-to-output highway networks. VC is a speech synthesis technique that converts input features into output speech parameters, and DNN-based acoustic models for VC are used to estimate the output speech parameters from the input speech parameters. Given that the input and output are often in the same domain (e.g., cepstrum) in VC, this paper proposes a VC using highway networks connected from the input to output. The acoustic models predict the weighted spectral differentials between the input and output spectral parameters. The architecture not only alleviates over-smoothing effects that degrade speech quality, but also effectively represents the characteristics of spectral parameters. The experimental results demonstrate that the proposed architecture outperforms Feed-Forward neural networks in terms of the speech quality and speaker individuality of the converted speech. key words: statistical parametric speech synthesis, DNN-based voice conversion, highway networks, over-smoothing

show abstract

Postfilters to Modify the Modulation Spectrum for Statistical Parametric Speech Synthesis

Cited by 62 publications

References 41 publications

A Neural Parametric Singing Synthesizer Modeling Timbre and Expression from Natural Songs

A Neural Parametric Singing Synthesizer Modeling Timbre and Expression from Natural Songs

The Voice Conversion Challenge 2016

Voice Conversion Using Input-to-Output Highway Networks

Contact Info

Product

Resources

About