UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation

Abstract

Most neural vocoders employ band-limited mel-spectrograms to generate waveforms. If full-band spectral features are used as the input, the vocoder can be provided with as much acoustic information as possible. However, in some models employing full-band mel-spectrograms, an over-smoothing problem occurs as part of which non-sharp spectrograms are generated. To address this problem, we propose UnivNet, a neural vocoder that synthesizes high-fidelity waveforms in real time. Inspired by works in the field of voice activity detection, we added a multiresolution spectrogram discriminator that employs multiple linear spectrogram magnitudes computed using various parameter sets. Using full-band mel-spectrograms as input, we expect to generate high-resolution signals by adding a discriminator that employs spectrograms of multiple resolutions as the input. In an evaluation on a dataset containing information on hundreds of speakers, UnivNet obtained the best objective and subjective results among competing models for both seen and unseen speakers. These results, including the best subjective score for text-to-speech, demonstrate the potential for fast adaptation to new speakers without a need for training from scratch.

RefineGAN: Universally Generating Waveform Better than Ground Truth with Highly Accurate Pitch and Intensity Responses

“…When trained with audio with a conventional sample rate (22050Hz or 24000Hz), evaluations had shown better audio quality and improved voice naturalness dimensions even when compared with flow-based models or autoregressive methods. Also, UnivNet [24] presented a new discriminator design based on the idea of deciding on a linear spectrogram calculated using STFT rather than on the waveform. This improvement dramatically reduces the discriminator's difficulty differentiating generated signals from ground truths and focuses more on the higher frequency component of the audio signal, which heavily affects the voice quality.…”

“…However, we use period parameters of 2,3,5,7,11 in the hope of being better suitable for 44100hz fullband generation and accelerating the training process by reducing the computational need for the discriminator. The other is the exact Multi-Resolution Discriminator we adopt from the UnviNet [24] and the same three different sets of parameters as the author mentioned in the paper. FFT sizes and hoping sizes in the three sets are perfectly compatible with the parameter sets we use for the Multi-param Mel Loss and thus avoid potential conflicts between them.…”

Phase-Aware Spoof Speech Detection Based on Res2Net with Phase Network

“…The spoofing attacks are mainly categorized into two types: physical access (PA) and logical access (LA). The PA considers the replay attack [2][3][4] while the LA includes the spoofing attacks based on text-to-speech synthesis [5][6][7][8] and voice conversion [9][10][11] technologies.…”

WOLONet: Wave Outlooker for Efficient and High Fidelity Speech Synthesis

“…When the generator reaches a Nash equalization point, it is expected to synthesize a high-quality waveform. GAN-based methods ( [2], [1], [21], [22], [23], [24], [3], [4], [25] etc.) are promising, as some models are even capable of synthesizing waves in realtime on a single GPU or even CPU while achieving a comparable MOS that is very suitable for actual industrial use.…”

“…Recently, HiFiGAN [3] proposed a novel multi-period discriminator and achieved the state of the art in wave quality and realtime speed at CPU. UnivNet [4] and Universal MelGAN [26] also propose multi-resolution spectrogram discriminators using a 2D convolution-based discriminator in the frequency domain to eliminate high-frequency artifacts, such as mental noise and reverberation in the auditory domain. StyleMelGAN [25] synthesizes high-quality waves by using the Adaptive Batch Normalization block conditioned by the Mel spectrogram.…”