An Experimental Analysis of Deep Learning Architectures for Supervised Speech Enhancement

Abstract

Recent speech enhancement research has shown that deep learning techniques are very effective in removing background noise. Many deep neural networks are being proposed, showing promising results for improving overall speech perception. The Deep Multilayer Perceptron, Convolutional Neural Networks, and the Denoising Autoencoder are well-established architectures for speech enhancement; however, choosing between different deep learning models has been mainly empirical. Consequently, a comparative analysis is needed between these three architecture types in order to show the factors affecting their performance. In this paper, this analysis is presented by comparing seven deep learning models that belong to these three categories. The comparison includes evaluating the performance in terms of the overall quality of the output speech using five objective evaluation metrics and a subjective evaluation with 23 listeners; the ability to deal with challenging noise conditions; generalization ability; complexity; and, processing time. Further analysis is then provided while using two different approaches. The first approach investigates how the performance is affected by changing network hyperparameters and the structure of the data, including the Lombard effect. While the second approach interprets the results by visualizing the spectrogram of the output layer of all the investigated models, and the spectrograms of the hidden layers of the convolutional neural network architecture. Finally, a general evaluation is performed for supervised deep learning-based speech enhancement while using SWOC analysis, to discuss the technique’s Strengths, Weaknesses, Opportunities, and Challenges. The results of this paper contribute to the understanding of how different deep neural networks perform the speech enhancement task, highlight the strengths and weaknesses of each architecture, and provide recommendations for achieving better performance. This work facilitates the development of better deep neural networks for speech enhancement in the future.

Two-Step Joint Optimization with Auxiliary Loss Function for Noise-Robust Speech Recognition

“…In recent years, the development of deep learning technology has substantially improved the performance of speech processing algorithms such as automatic speech recognition (ASR) [ 1 , 2 ], speech separation [ 3 ], and speech enhancement [ 4 ]. Among them, ASR has been popularly deployed for voice-enabled information retrieval using artificial intelligence (AI) speakers and chatbots [ 5 , 6 , 7 , 8 ].…”

“…Traditionally, speech enhancement techniques have been developed to enhance speech quality for voice communications equipped with a single-channel microphone or a multi-channel microphone array [ 4 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 ]. To overcome the noise robustness of ASR, the developed speech enhancement algorithm can be used as a front-end of ASR.…”

“…To overcome the noise robustness of ASR, the developed speech enhancement algorithm can be used as a front-end of ASR. Among various types of speech enhancement algorithms, deep learning-based speech enhancement models achieved a superior performance compared to conventional statistical methods [ 4 ]. In particular, U-Net-based speech enhancement models showed better performance than other neural network architectures [ 28 ].…”

Enhancement of single channel speech quality and intelligibility in multiple noise conditions using wiener filter and deep CNN

“…The data-driven approach (Zhao et al 2018) of the deep neural network makes it more efficient and is responsive to untrained conditions and unseen noises. In the recent past, the commonly used techniques for supervised speech enhancement (Nossier et al 2021) technique include the mapping in the frequency domain or time-frequency masking. The speech signal is converted from the frequency domain to the time domain.…”

A Fully Connected Deep Neural Network approach with multiple sub-frame consideration and phase recompense for noise suppression

“…Nossier et al [20] have demonstrated a comparative analysis on the basis of three classes including the initially proposed Deep Multi-layer Perceptron (MLP), Convolutional Neural Networks (CNN), and Denoising Autoencoder (DAE). The work carried out investigates the impact of network hyperparameter changes and data arrangement on the performance together with the Lombard effect.…”