Deep Speech Enhancement for Reverberated and Noisy Signals using Wide Residual Networks

Abstract

This paper proposes a deep speech enhancement method which exploits the high potential of residual connections in a wide neural network architecture, a topology known as Wide Residual Network. This is supported on single dimensional convolutions computed alongside the time domain, which is a powerful approach to process contextually correlated representations through the temporal domain, such as speech feature sequences. We find the residual mechanism extremely useful for the enhancement task since the signal always has a linear shortcut and the non-linear path enhances it in several steps by adding or subtracting corrections. The enhancement capacity of the proposal is assessed by objective quality metrics and the performance of a speech recognition system. This was evaluated in the framework of the REVERB Challenge dataset, including simulated and real samples of reverberated and noisy speech signals. Results showed that enhanced speech from the proposed method succeeded for both, the enhancement task with intelligibility purposes and the speech recognition system. The DNN model, trained with artificial synthesized reverberation data, was able to deal with far-field reverberated speech from real scenarios. Furthermore, the method was able to take advantage of the residual connection achieving to enhance signals with low noise level, which is usually a strong handicap of traditional enhancement methods.

Magnitude or Phase? A Two Stage Algorithm for Dereverberation

“…In order to direct the RI2RI model to focus on the phase estimation task, it was pre-trained with a synthetic data constructed with clean magnitude and noisy phase. The pre-training of this model also utilizes (5).…”

“…In recent years, deep neural network (DNN)-based models were utilized to deal with this challenge. The majority of these methods attempts to enhance the magnitude of the noisy and reveberant short time Fourier transform (STFT) [3][4][5]. In these approaches, the enhanced magnitude is combined with the noisy phase and then inverse-transformed to the timedomain.…”

Complex-Valued Time-Frequency Self-Attention for Speech Dereverberation

“…Additionally, advances in technologies such as hearing aids require the speech systems to enhance perceptual quality of speech captured in adverse environmental conditions, thus improving human hearing abilities. Several deep learning (DL)-based speech enhancement systems have been successfully developed to address concurrent improvements in perceptual quality and performance of back-end speech and language applications using fully convolutional neural networks (FCN), and recurrent networks (RNN) [9,10,11,12]. The majority of these approaches work with the complex short-term fourier transform (STFT) of distorted speech, either to enhance the log-power spectrum (LPS) and reuse the unaltered distorted phase signal [13,14,15,16,17], or to estimate the complex ratio mask (cRM) [18,19,20] and directly enhance the complex spectrogram to restore a cleaner time-domain signal.…”

“…As deep neural networks (DNN) advance to be compatible with complex representations, researchers have investigated many speech enhancement strategies to estimate cRM using deep complex neural networks (DCNN). To address reverberation which distorts the signal in both time and frequency, many sequence-to-sequence learning strategies such as recurrent neural networks (RNNs) and long short-term memory (LSTM) [21,11] have also been explored. In addition to the FCNs, these methods capture and leverage the temporal correlations for speech dereverberation.…”

On the Use of Audio Fingerprinting Features for Speech Enhancement with Generative Adversarial Network

“…Frequency-domain features such as STFT, Gammatone spectrum and Mel-Frequency Cepstral Coefficients (MFCC) have been used frequently. In addition, a combination of STFT with MFCC is employed in [9] for training wide residual networks for speech enhancement. Compared to STFT, filterbased features like MFCC exhibit reduced dimensionality and are more suitable for learning algorithms, as they can reduce memory and computational requirements while maintaining comparable level of performance [7], [10], [11].…”