GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis

Abstract

Recent advances in neural multi-speaker text-to-speech (TTS) models have enabled the generation of reasonably good speech quality with a single model and made it possible to synthesize the speech of a speaker with limited training data. Finetuning to the target speaker data with the multi-speaker model can achieve better quality, however, there still exists a gap compared to the real speech sample and the model depends on the speaker. In this work, we propose GANSpeech, which is a high-fidelity multi-speaker TTS model that adopts the adversarial training method to a non-autoregressive multi-speaker TTS model. In addition, we propose simple but efficient automatic scaling methods for feature matching loss used in adversarial training. In the subjective listening tests, GANSpeech significantly outperformed the baseline multi-speaker FastSpeech and FastSpeech2 models, and showed a better MOS score than the speaker-specific fine-tuned FastSpeech2.

MixGAN-TTS: Efficient and Stable Speech Synthesis Based on Diffusion Model

“…The discriminator structure is modeled and represented by D ϕ (x t−1 , x t , t, s) with learnable parameters ϕ. The discriminator uses joint conditional and unconditional loss (JCU) [26], which combines conditional and unconditional adversarial losses to further improve the accuracy of the mel-spectrogram and speech waveform mapping.…”

Research on Speech Synthesis Based on Mixture Alignment Mechanism

“…In addition, speech synthesis has introduced some other generative models that have also achieved very good performance. Flow-based models are found in [ 15 , 16 , 17 ], variational autoencoder (VAE)-based models are listed in [ 17 , 18 ], generative adversarial network (GAN)-based models are presented in [ 19 ], and diffusion process-based models are described in [ 20 , 21 , 22 , 23 ].…”