Monaural Speech Enhancement with Complex Convolutional Block Attention Module and Joint Time Frequency Losses

Zhao, Shengkui; Nguyen, Trung Hieu; Ma, Bin

doi:10.1109/icassp39728.2021.9414569

Cited by 39 publications

(27 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The skip connections facilitate optimization by connecting each block in the encoder to its corresponding block in the decoder. We further add the attention block CCBAM [7] on the skip pathway to facilitate information flow. Note that we take all convolutions to be causal in time by applying asymmetrical paddings.…”

Section: The Overall Architecturementioning

confidence: 99%

“…It is a signal-level loss and directly performs on the signal itself. To further guide the learning of cIRM estimation, we also considered the mean squared error (MSE) losses of the real Mr and imaginary Mi estimates of cIRM [7]. Specifically, we optimize the FRCRN model by the following joint loss function:…”

Section: Joint Loss Functionmentioning

confidence: 99%

“…The CRN has been shown to be very effective for speech enhancement [4,5] and the extended complex-valued DCCRN [6] achieved the best performance in the Interspeech 2020 DNS Challenge. In [7], it was demonstrated that CRN heavily relies on the representation power of the convolutions in CED and a complex convolutional block attention module (CCBAM) was introduced to boost the feature representation, resulting in improved performance. However, due to the limited receptive fields of convolutions, CRN still cannot capture well the long-range correlations along the frequency axis.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

FRCRN: Boosting Feature Representation Using Frequency Recurrence for Monaural Speech Enhancement

Zhao

Watcharasupat

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Convolutional recurrent networks (CRN) integrating a convolutional encoder-decoder (CED) structure and a recurrent structure have achieved promising performance for monaural speech enhancement. However, feature representation across frequency context is highly constrained due to limited receptive fields in the convolutions of CED. In this paper, we propose a convolutional recurrent encoderdecoder (CRED) structure to boost feature representation along the frequency axis. The CRED applies frequency recurrence on 3D convolutional feature maps along the frequency axis following each convolution, therefore, it is capable of catching long-range frequency correlations and enhancing feature representations of speech inputs. The proposed frequency recurrence is realized efficiently using a feedforward sequential memory network (FSMN). Besides the CRED, we insert two stacked FSMN layers between the encoder and the decoder to model further temporal dynamics. We name the proposed framework as Frequency Recurrent CRN (FRCRN). We design FRCRN to predict complex Ideal Ratio Mask (cIRM) in complex-valued domain and optimize FRCRN using both time-frequency-domain and time-domain losses. Our proposed approach achieved state-of-the-art performance on wideband benchmark datasets and achieved 2nd place for the real-time fullband track in terms of Mean Opinion Score (MOS) and Word Accuracy (WAcc) in the ICASSP 2022 Deep Noise Suppression (DNS) challenge.

show abstract

Section: The Overall Architecturementioning

confidence: 99%

Section: Joint Loss Functionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FRCRN: Boosting Feature Representation Using Frequency Recurrence for Monaural Speech Enhancement

Zhao

Watcharasupat

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The baseline systems used for performance comparison are Optimally Modified Log Spectral Amplitude (OMLSA) method [23], attention-wave-U-net [24], Attention-DCUNet [25], and TSTNN [26]. The setup of the baseline systems are detailed as below:…”

Section: Baselinesmentioning

confidence: 99%

U-Former: Improving Monaural Speech Enhancement with Multi-head Self and Cross Attention

Xu¹,

Hao²

2022

Preprint

View full text Add to dashboard Cite

For supervised speech enhancement, contextual information is important for accurate spectral mapping. However, commonly used deep neural networks (DNNs) are limited in capturing temporal contexts. To leverage long-term contexts for tracking a target speaker, this paper treats the speech enhancement as sequence-to-sequence mapping, and propose a novel monaural speech enhancement U-net structure based on Transformer, dubbed U-Former. The key idea is to model long-term correlations and dependencies, which are crucial for accurate noisy speech modeling, through the multi-head attention mechanisms. For this purpose, U-Former incorporates multi-head attention mechanisms at two levels: 1) a multi-head self-attention module which calculate the attention map along both timeand frequency-axis to generate time and frequency sub-attention maps for leveraging global interactions between encoder features, while 2) multi-head cross-attention module which are inserted in the skip connections allows a fine recovery in the decoder by filtering out uncorrelated features. Experimental results illustrate that the U-Former obtains consistently better performance than recent models of PESQ, STOI, and SSNR scores.

show abstract

“…Inspired by a recent work demonstrating that DNNs implementing complex operators [58] may outperform previous architectures in many audiorelated tasks, new state-of-the-art performances were achieved on speech enhancement using complex representations of audio data [14,15]. Recent work was able to further improve these approaches by introducing a complex convolutional block attention module (CCBAM) and a mixed loss function [59].…”

Section: Audio Enhancementmentioning

confidence: 99%

Stochastic Restoration of Heavily Compressed Musical Audio Using Generative Adversarial Networks

Lattner¹,

Nistal²

2021

Electronics

View full text Add to dashboard Cite

Lossy audio codecs compress (and decompress) digital audio streams by removing information that tends to be inaudible in human perception. Under high compression rates, such codecs may introduce a variety of impairments in the audio signal. Many works have tackled the problem of audio enhancement and compression artifact removal using deep-learning techniques. However, only a few works tackle the restoration of heavily compressed audio signals in the musical domain. In such a scenario, there is no unique solution for the restoration of the original signal. Therefore, in this study, we test a stochastic generator of a Generative Adversarial Network (GAN) architecture for this task. Such a stochastic generator, conditioned on highly compressed musical audio signals, could one day generate outputs indistinguishable from high-quality releases. Therefore, the present study may yield insights into more efficient musical data storage and transmission. We train stochastic and deterministic generators on MP3-compressed audio signals with 16, 32, and 64 kbit/s. We perform an extensive evaluation of the different experiments utilizing objective metrics and listening tests. We find that the models can improve the quality of the audio signals over the MP3 versions for 16 and 32 kbit/s and that the stochastic generators are capable of generating outputs that are closer to the original signals than those of the deterministic generators.

show abstract

Monaural Speech Enhancement with Complex Convolutional Block Attention Module and Joint Time Frequency Losses

Cited by 39 publications

References 20 publications

FRCRN: Boosting Feature Representation Using Frequency Recurrence for Monaural Speech Enhancement

FRCRN: Boosting Feature Representation Using Frequency Recurrence for Monaural Speech Enhancement

U-Former: Improving Monaural Speech Enhancement with Multi-head Self and Cross Attention

Stochastic Restoration of Heavily Compressed Musical Audio Using Generative Adversarial Networks

Contact Info

Product

Resources

About