Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities

Abstract

Voice conversion (VC) using sequence-to-sequence learning of context posterior probabilities is proposed. Conventional VC using shared context posterior probabilities predicts target speech parameters from the context posterior probabilities estimated from the source speech parameters. Although conventional VC can be built from non-parallel data, it is difficult to convert speaker individuality such as phonetic property and speaking rate contained in the posterior probabilities because the source posterior probabilities are directly used for predicting target speech parameters. In this work, we assume that the training data partly include parallel speech data and propose sequence-to-sequence learning between the source and target posterior probabilities. The conversion models perform non-linear and variable-length transformation from the source probability sequence to the target one. Further, we propose a joint training algorithm for the modules. In contrast to conventional VC, which separately trains the speech recognition that estimates posterior probabilities and the speech synthesis that predicts target speech parameters, our proposed method jointly trains these modules along with the proposed probability conversion modules. Experimental results demonstrate that our approach outperforms the conventional VC.

Investigation of F0 Conditioning and Fully Convolutional Networks in Variational Autoencoder Based Voice Conversion

“…Note that here the output of our model is still of the same length as the input. Although sequence to sequence based models, which can generate output sequences of variable length, have been successfully applied to VC [20,21,22,23,24], we will show that only considering temporal dependencies can bring significant improvements to VAE-VC.…”

Introduction to Voice Presentation Attack Detection and Recent Advances

“…Recently, deep learning has changed the above standard procedures for voice conversion and we can see many different solutions now. For instance, variational auto-encoder or sequence-to-sequence neural networks enable us to build VC systems without using frame level alignment [102,103]. It has also been showed that a cycle-consistent adversarial network called "CycleGAN" [104] is one possible solution for building VC systems without using a parallel corpus.…”

One-Shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization

“…VC aims to convert the non-linguistic information of the speech signals while maintaining the linguistic content the same. The non-linguistic information may refer to speaker identity [1,2,3], accent or pronunciation [4,5] to name a few. VC can be useful in some down-stream tasks like multi-speaker text-tospeech [6,7] and expressive speech synthesis [8,9], and also some applications like speech enhancement [10,11,12] or pronunciation correction [4], and so on.…”