Towards Achieving Robust Universal Neural Vocoding

Abstract

This paper explores the potential universality of neural vocoders. We train a WaveRNN-based vocoder on 74 speakers coming from 17 languages. This vocoder is shown to be capable of generating speech of consistently good quality (98% relative mean MUSHRA when compared to natural speech) regardless of whether the input spectrogram comes from a speaker or style seen during training or from an out-of-domain scenario when the recording conditions are studio-quality. When the recordings show significant changes in quality, or when moving towards non-speech vocalizations or singing, the vocoder still significantly outperforms speaker-dependent vocoders, but operates at a lower average relative MUSHRA of 75%. These results are shown to be consistent across languages, regardless of them being seen during training (e.g. English or Japanese) or unseen (e.g. Wolof, Swahili, Ahmaric).

An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning

“…4) Recent Progress on Neural Vocoders: More recently, speaker independent WaveRNN-based neural vocoder [63] became popular as it can generate human-like voices from both in-domain and out-of-domain spectrogram [101]- [103]. Another well-known neural vocoder that achieves high-quality synthesis performance is WaveGlow [64].…”

Privacy-preserving Voice Analysis via Disentangled Representations

“…Neural vocoders such as Wavenet [62] have rapidly become the most commonly used vocoding method for speech synthesis. Although it improved the quality of generated speech, it has significant cost in computation power and data sources, and suffers from poor generalization [50]. To solve this problem, many architectures such as Wave Recurrent Neural Networks (WaveRNN) [36] have been proposed.…”

“…WaveRNN combines linear prediction with recurrent neural networks to synthesize neural audio much faster than other neural synthesizers. In our framework, we use WaveRNN as a decoder with a minor change suggested by [50]. The autoregressive component consists of a single forward gated recurrent unit (GRU) (hidden size of 896) and a pair of affine layers followed by a softmax layer with 1024 outputs, predicting the 10-bit mu-law samples for a 24 kHz sampling rate.…”

BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimization

“…Model weights are tuned with an ADAM optimizer to minimize the teacher-forced L1 loss between predicted and extracted mel-spectrograms. To complete the TTS pipeline, we convert mel-spectograms to waveforms using the multi-speaker neural vocoder of [19]. This vocoder is trained across 74 speakers and suitable for generating natural speech for our wide-range of adaptation speakers.…”