High-quality Speech Synthesis Using Super-resolution Mel-Spectrogram

Sheng, Leyuan; Huang, Dazhi; Pavlovskiy, Evgeniy

doi:10.48550/arxiv.1912.01167

Cited by 4 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by the human hearing perception, the MFCC has emerged as a widely accepted method in acoustic signal processing. However, the discrete cosine transform (DCT) in the MFCC approach may filter out a substantial amount of valuable data [39]. Deep learning has a powerful ability to extract features, and researchers prefer to extract features from the Mel spectrogram which contains more information.…”

Section: Feature Preparingmentioning

confidence: 99%

A Lightweight Network Based on Multi-Scale Asymmetric Convolutional Neural Networks with Attention Mechanism for Ship-Radiated Noise Classification

Yan,

Yao

et al. 2024

JMSE

View full text Add to dashboard Cite

Ship-radiated noise classification is critical in ocean acoustics. Recently, the feature extraction method combined with time–frequency spectrograms and convolutional neural networks (CNNs) has effectively described the differences between various underwater targets. However, many existing CNNs are challenging to apply to embedded devices because of their high computational costs. This paper introduces a lightweight network based on multi-scale asymmetric CNNs with an attention mechanism (MA-CNN-A) for ship-radiated noise classification. Specifically, according to the multi-resolution analysis relying on the relationship between multi-scale convolution kernels and feature maps, MA-CNN-A can autonomously extract more fine-grained multi-scale features from the time–frequency domain. Meanwhile, the MA-CNN-A maintains its light weight by employing asymmetric convolutions to balance accuracy and efficiency. The number of parameters introduced by the attention mechanism only accounts for 0.02‰ of the model parameters. Experiments on the DeepShip dataset demonstrate that the MA-CNN-A outperforms some state-of-the-art networks with a recognition accuracy of 98.2% and significantly decreases the parameters. Compared with the CNN based on three-scale square convolutions, our method has a 68.1% reduction in parameters with improved recognition accuracy. The results of ablation explorations prove that the improvements benefit from asymmetric convolution, multi-scale block, and attention mechanism. Additionally, MA-CNN-A shows a robust performance against various interferences.

show abstract

Section: Feature Preparingmentioning

confidence: 99%

A Lightweight Network Based on Multi-Scale Asymmetric Convolutional Neural Networks with Attention Mechanism for Ship-Radiated Noise Classification

Yan,

Yao

et al. 2024

JMSE

View full text Add to dashboard Cite

show abstract

“…The authors in [13] introduce an end-toend GAN based system for speech bandwidth extension for use in downstream automatic speech recognition. The authors in [14] suggest to use a WaveNet model to directly output a high sampled speech signal while the authors in [15] suggest using GANs to estimate the mel-spectrogram and then apply a vocoder to generate enhanced waveform.…”

Section: Related Workmentioning

confidence: 99%

High Fidelity Speech Regeneration with Application to Speech Enhancement

Polyak¹,

Wolf²,

Adi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Speech enhancement has seen great improvement in recent years mainly through contributions in denoising, speaker separation, and dereverberation methods that mostly deal with environmental effects on vocal audio. To enhance speech beyond the limitations of the original signal, we take a regeneration approach, in which we recreate the speech from its essence, including the semi-recognized speech, prosody features, and identity. We propose a wav-to-wav generative model for speech that can generate 24khz speech in a realtime manner and which utilizes a compact speech representation, composed of ASR and identity features, to achieve a higher level of intelligibility. Inspired by voice conversion methods, we train to augment the speech characteristics while preserving the identity of the source using an auxiliary identity network. Perceptual acoustic metrics and subjective tests show that the method obtains valuable improvements over recent baselines.

show abstract

“…In [31], the authors applied the adversarial training to an super-resolution network converting mel spectrogram that generated by the AR mel-synthesis network to linear spectrogram. In addition, in [32], the authors found that the mel spectrogram generated by the TTS model trained with only reconstruction loss was over-smooth, and proposed an enhancer model for mel spectrogram that reduces the gap between the true mel spectrogram and the generated mel spectrogram.…”

Section: Introductionmentioning

confidence: 99%

GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis

Yang¹,

Bae²,

Bak³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent advances in neural multi-speaker text-to-speech (TTS) models have enabled the generation of reasonably good speech quality with a single model and made it possible to synthesize the speech of a speaker with limited training data. Finetuning to the target speaker data with the multi-speaker model can achieve better quality, however, there still exists a gap compared to the real speech sample and the model depends on the speaker. In this work, we propose GANSpeech, which is a high-fidelity multi-speaker TTS model that adopts the adversarial training method to a non-autoregressive multi-speaker TTS model. In addition, we propose simple but efficient automatic scaling methods for feature matching loss used in adversarial training. In the subjective listening tests, GANSpeech significantly outperformed the baseline multi-speaker FastSpeech and FastSpeech2 models, and showed a better MOS score than the speaker-specific fine-tuned FastSpeech2.

show abstract

High-quality Speech Synthesis Using Super-resolution Mel-Spectrogram

Cited by 4 publications

References 27 publications

A Lightweight Network Based on Multi-Scale Asymmetric Convolutional Neural Networks with Attention Mechanism for Ship-Radiated Noise Classification

A Lightweight Network Based on Multi-Scale Asymmetric Convolutional Neural Networks with Attention Mechanism for Ship-Radiated Noise Classification

High Fidelity Speech Regeneration with Application to Speech Enhancement

GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis

Contact Info

Product

Resources

About