Statistical Voice Conversion with WaveNet-Based Waveform Generation

Kobayashi, Kenzo; Hayashi, Tomoki; Tamamori, Akira; Toda, Tomoki

doi:10.21437/interspeech.2017-986

Cited by 78 publications

(73 citation statements)

References 23 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, human perception is quite sensitive to speech quality, and that of synthesized speech highly depends on the generation model. WaveNet (WN) [4] is one of the state-of-the-art speech generation models, which has been applied to many applications, such as speech enhancement [12,13], text-to-speech (TTS) [7,9], speech coding [11], and voice conversion (VC) [15][16][17][18]. Specifically, WN is an autoregressive model that predicts a current speech sample based on a specific number of previous samples which is called the receptive field.…”

Section: Introductionmentioning

confidence: 99%

Quasi-Periodic WaveNet Vocoder: A Pitch Dependent Dilated Convolution Model for Parametric Speech Generation

Hayashi

Tobing

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a quasi-periodic neural network (QPNet) vocoder with a novel network architecture named pitchdependent dilated convolution (PDCNN) to improve the pitch controllability of WaveNet (WN) vocoder. The effectiveness of the WN vocoder to generate high-fidelity speech samples from given acoustic features has been proved recently. However, because of the fixed dilated convolution and generic network architecture, the WN vocoder hardly generates speech with given F0 values which are outside the range observed in training data. Consequently, the WN vocoder lacks the pitch controllability which is one of the essential capabilities of conventional vocoders. To address this limitation, we propose the PDCNN component which has the time-variant adaptive dilation size related to the given F0 values and a cascade network structure of the QPNet vocoder to generate quasi-periodic signals such as speech. Both objective and subjective tests are conducted, and the experimental results demonstrate the better pitch controllability of the QPNet vocoder compared to the same and double-size WN vocoders while attaining comparable speech qualities.

show abstract

Section: Introductionmentioning

confidence: 99%

Quasi-Periodic WaveNet Vocoder: A Pitch Dependent Dilated Convolution Model for Parametric Speech Generation

Hayashi

Tobing

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…WaveNet [18] as one of the state-of-the-art audio generation models has been widely applied to various VC systems that take WN as a vocoder to generate converted waveforms from the converted acoustic features. For example, Kobayashi et al [28] combined GMM-based Mel-cepstral coefficient (mcep) conversion and linear transformation of prosodic features with the WN vocoder. Furthermore, in our previous works, we explored the effectiveness of different mcep conversion models with the WN vocoder, including a DNN [25,29], deep mixture density network (DMDN) [26], VAE [30], long short-term memory (LSTM) [31], and gated recurrent unit (GRU) [32].…”

Section: Related Workmentioning

confidence: 99%

Statistical Voice Conversion with Quasi-periodic WaveNet Vocoder

Wu¹,

Tobing²,

Hayashi³

et al. 2019

10th ISCA Workshop on Speech Synthesis (SSW 10)

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate the effectiveness of a quasi-periodic WaveNet (QPNet) vocoder combined with a statistical spectral conversion technique for a voice conversion task. The WaveNet (WN) vocoder has been applied as the waveform generation module in many different voice conversion frameworks and achieves significant improvement over conventional vocoders. However, because of the fixed dilated convolution and generic network architecture, the WN vocoder lacks robustness against unseen input features and often requires a huge network size to achieve acceptable speech quality. Such limitations usually lead to performance degradation in the voice conversion task. To overcome this problem, the QPNet vocoder is applied, which includes a pitch-dependent dilated convolution component to enhance the pitch controllability and attain a more compact network than the WN vocoder. In the proposed method, input spectral features are first converted using a framewise deep neural network, and then the QPNet vocoder generates converted speech conditioned on the linearly converted prosodic and transformed spectral features. The experimental results confirm that the QPNet vocoder achieves significantly better performance than the same-size WN vocoder while maintaining comparable speech quality to the double-size WN vocoder.

show abstract

“…The NU VC system uses a WaveNet-based vocoder [17,18,19] to model the waveform of the target speaker and generate the converted waveform using estimated speech features. Several flows are used in producing the estimated spectral features, where the direct waveform modification [2] method is employed.…”

Section: Waveform-processing Modulementioning

confidence: 99%

“…On the other hand, in the handling of prosodic parameters, such as fundamental frequency (F0), several methods have been commonly used including a simple mean/variance linear transformation, a contour-based transformation [13], GMM-based mapping [14], and neural network [15]. For waveform generation, approaches include the source-filter vocoder system [16], the latest direct waveform modification technique [2], and the use of state-ofthe-art WaveNet modeling [17,18,19].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

NU Voice Conversion System for the Voice Conversion Challenge 2018

Tobing¹,

Wu²,

Hayashi³

et al. 2018

EasyChair Preprints

Self Cite

View full text Add to dashboard Cite

This paper presents the NU (Nagoya University) voice conversion (VC) system for the HUB task of the Voice Conversion Challenge 2018 (VCC 2018). The design of the NU VC system can basically be separated into two modules consisting of a speech parameter conversion module and a waveformprocessing module. In the speech parameter conversion module, a deep learning framework is deployed to estimate the spectral parameters of a target speaker given those of a source speaker. Specifically, a deep neural network (DNN) and a deep mixture density network (DMDN) are used as the deep model structure. In the waveform-processing module, given the estimated spectral parameters and linearly transformed F0 parameters, the converted waveform is generated using a WaveNet-based vocoder system. To use the WaveNet-based vocoder, there are several generation flows based on an analysissynthesis framework to obtain the speech parameter set, on the basis of which a system selection process is performed to select the best one in an utterance-wise manner. The results of VCC 2018 ranked the NU VC system in second place with an overall mean opinion score (MOS) of 3.44 for speech quality and 85% accuracy for speaker similarity.

show abstract

Statistical Voice Conversion with WaveNet-Based Waveform Generation

Cited by 78 publications

References 23 publications

Quasi-Periodic WaveNet Vocoder: A Pitch Dependent Dilated Convolution Model for Parametric Speech Generation

Quasi-Periodic WaveNet Vocoder: A Pitch Dependent Dilated Convolution Model for Parametric Speech Generation

Statistical Voice Conversion with Quasi-periodic WaveNet Vocoder

NU Voice Conversion System for the Voice Conversion Challenge 2018

Contact Info

Product

Resources

About