A Multi-Scale Feature Recalibration Network for End-to-End Single Channel Speech Enhancement

Xian, Yang; Sun, Yang; Wang, Wenwu; Naqvi, Syed Mohsen

doi:10.1109/jstsp.2020.3045846

Cited by 11 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In speech enhancement problem, the Short-time Fourier transform (STFT) is widely used [14], [15], [16] to achieve feature extraction. The main idea is supported by the recent research in speech processing that it is not clear what cues in which scales of window lengths contribute most to the final performance [8].…”

Section: Related Workmentioning

confidence: 99%

Self-Supervised Learning based Monaural Speech Enhancement with Complex-Cycle-Consistent

Li¹,

Sun²,

Naqvi³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, self-supervised learning (SSL) techniques have been introduced to solve the monaural speech enhancement problem. Due to the lack of using clean phase information, the enhancement performance is limited in most SSL methods. Therefore, in this paper, we propose a phase-aware self-supervised learning based monaural speech enhancement method. The latent representations of both amplitude and phase are studied in two decoders of the foundation autoencoder (FAE) with only a limited set of clean speech signals independently. Then, the downstream autoencoder (DAE) learns a shared latent space between the clean speech and mixture representations with a large number of unseen mixtures. A complex-cycle-consistent (CCC) mechanism is proposed to minimize the reconstruction loss between the amplitude and phase domains. Besides, it is noticed that if the speech features are extracted as the multi-resolution spectra, the desired information distributed in spectra of different scales can be studied to further boost the performance. The NOISEX and DAPS corpora are used to generate mixtures with different interferences to evaluate the efficacy of the proposed method. It is highlighted that the clean speech and mixtures fed in FAE and DAE are not paired. Both ablation and comparison experimental results show that the proposed method clearly outperforms the state-of-the-art approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Self-Supervised Learning based Monaural Speech Enhancement with Complex-Cycle-Consistent

Li¹,

Sun²,

Naqvi³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Numerous approaches have been presented for extracting features at multiple scales that improve the quality of speech. In [15], the author proposed a multiscale feature network that uses different-sized convolution kernels to extract features at multiple scales. Due to the massive convolution kernel, the model has a large number of trainable parameters.…”

Section: Introductionmentioning

confidence: 99%

Single channel speech enhancement using time-frequency attention mechanism based nested U-net model

Prathipati,

Chakravarthy

2024

Eng. Res. Express

View full text Add to dashboard Cite

Deep-learning models have used attention mechanisms to improve the quality and intelligibility of noisy speech, demonstrating the effectiveness of attention mechanisms. We rely on either spatial or temporal-based attention mechanisms, resulting in severe information loss. In this paper, a time-frequency attention mechanism with a nested U-network (TFANUNet) is proposed for single-channel speech enhancement. By using time-frequency attention (TFA), learns the channel, frequency and time information which is more significant for speech enhancement. Basically, the proposed model is an encoder-decoder model, where each layer in the encoder and decoder is followed by a nested dense residual dilated DensNet (NDRD) based multi-scale context aggression block. NDRD involves multiple dilated convolution with different dilatation factors to explore the large receptive area at different scales simultaneously. NDRD avoids the aliasing problem in DenseNet. We integrated the TFA and NDRD blocks into the proposed model to enable refined feature set extraction without information loss and utterance-level context aggregation, respectively. The proposed TFANUNet model results outperform baselines in terms of STOI and PESQ.

show abstract

“…Considering the processing domain, traditional speech-enhancement methods can be divided into three categories, namely time [6][7][8][9][10], frequency [11][12][13][14] and time-frequency [15][16][17][18] domain methods. Despite the significant development of speech-enhancement techniques [19][20][21][22][23][24][25] from particular categories in recent years, there is still a problem of obtaining a high-quality speech signal with high noise attenuation. Typically, if a method maintains high noise attenuation properties, such a method does not always provide a significant improvement in speech quality.…”

Section: Introductionmentioning

confidence: 99%

Speech Enhancement Based on Enhanced Empirical Wavelet Transform and Teager Energy Operator

Kuwałek

Jęśko

2023

Electronics

View full text Add to dashboard Cite

This paper presents a new speech-enhancement approach based on an enhanced empirical wavelet transform, considering the time and scale adaptation of thresholds for individual component signals obtained from the used transform. The time adaptation is performed using the Teager energy operator on the individual component signals, and the scale adaptation of thresholds is performed by the modified level-dependent threshold principle for the individual component signals. The proposed approach does not require an explicit estimation of the noise level or a priori knowledge of the signal-to-noise ratio as is usually needed in most common speech-enhancement methods. The effectiveness of the proposed method has been assessed based on over 1000 speech recordings from the public Librispeech database. The research included various types of noise (among others white, violet, brown, blue, and pink) and various types of disturbance (among others traffic sounds, hair dryer, and fan), which were added to the selected test signals. The score of perceptual evaluation of speech quality, allowing for the assessment of the quality of enhanced speech, and signal-to-noise ratio, allowing for the assessment of the effectiveness of disturbance attenuation, are selected for the evaluation of the resultant effectiveness of the proposed approach. The resultant effectiveness of the proposed approach is compared with other selected speech-enhancement methods or denoising techniques available in the literature. The experimental research results show that the proposed method performs better than conventional methods in many types of high-noise conditions in terms of producing less residual noise and lower speech distortion.

show abstract

A Multi-Scale Feature Recalibration Network for End-to-End Single Channel Speech Enhancement

Cited by 11 publications

References 31 publications

Self-Supervised Learning based Monaural Speech Enhancement with Complex-Cycle-Consistent

Self-Supervised Learning based Monaural Speech Enhancement with Complex-Cycle-Consistent

Single channel speech enhancement using time-frequency attention mechanism based nested U-net model

Speech Enhancement Based on Enhanced Empirical Wavelet Transform and Teager Energy Operator

Contact Info

Product

Resources

About