A Survey on Neural Speech Synthesis

Abstract

Text to speech (TTS), or speech synthesis, which aims to synthesize intelligible and natural speech given text, is a hot research topic in speech, language, and machine learning communities and has broad applications in the industry. As the development of deep learning and artificial intelligence, neural network-based TTS has significantly improved the quality of synthesized speech in recent years. In this paper, we conduct a comprehensive survey on neural TTS, aiming to provide a good understanding of current research and future trends. We focus on the key components in neural TTS, including text analysis, acoustic models, and vocoders, and several advanced topics, including fast TTS, low-resource TTS, robust TTS, expressive TTS, and adaptive TTS, etc. We further summarize resources related to TTS (e.g., datasets, opensource implementations) and discuss future research directions. This survey can serve both academic researchers and industry practitioners working on TTS.

YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone

“…This was done because although the latest works in ZS-TTS [3,4,10] only use the VCTK dataset, this dataset has a limited number of speakers (109) and little variety of recording conditions. Thus, after training with only this dataset, in general, ZS-TTS models do not generalize satisfactorily to new speakers where recording conditions or voice characteristics are very different than those seen in the training [12].…”

“…The different recording conditions are a challenge for the generalization of the zero-shot multi-speaker TTS models. In addition, speakers who have a voice that differs greatly from those seen in training also become a challenge [12]. Nevertheless, to show the potential of our model for adaptation to new speakers/recording conditions, we selected from 20 to 61 seconds of speech for 2 speakers (1M/1F) from Portuguese and the same for English in the Common Voice [37] dataset.…”

“…Also, we would like to thank the Defined.ai 11 for making industrial-level MOS testing so easily available. Finally, we would like to thank all contributors to the Coqui TTS repository 12 , this work was only possible thanks to the commitment of all.…”

“…ZS-TTS models still require a large number of speakers for training, making it difficult to obtain highquality models in low-resource languages. Furthermore, according to [12], the quality of current ZS-TTS models is not sufficiently good, especially for target speakers with speech characteristics that differ from those seen in training. Although SC-GlowTTS [10] achieved promising results with only 11 speakers from the VCTK dataset [13], generally, limiting the number and variety of training speakers further hinders the generalization of the model for unseen voices.…”

KazakhTTS2: Extending the Open-Source Kazakh TTS Corpus With More Data, Speakers, and Topics

“…Considering the aforementioned benefits, TTS is undoubtedly an essential speech processing technology for any language. In recent years, TTS research has progressed remarkably thanks to neural network-based architectures (Tan et al, 2021), regularly organized challenges (Black and Tokuda, 2005;Dunbar et al, 2019), and open-source datasets (Ito and Johnson, 2017;Zen et al, 2019;Shi et al, 2020). Especially, impressive results have been achieved for commercially viable languages, such as English and Mandarin.…”