Speech Enhancement Based on Denoising Autoencoder With Multi-Branched Encoders

Abstract

Deep learning-based models have greatly advanced the performance of speech enhancement (SE) systems. However, two problems remain unsolved, which are closely related to model generalizability to noisy conditions: (1) mismatched noisy condition during testing, i.e., the performance is generally suboptimal when models are tested with unseen noise types that are not involved in the training data; (2) local focus on specific noisy conditions, i.e., models trained using multiple types of noises cannot optimally remove a specific noise type even though the noise type has been involved in the training data. These problems are common in real applications. In this paper, we propose a novel denoising autoencoder with a multi-branched encoder (termed DAEME) model to deal with these two problems. In the DAEME model, two stages are involved: training and testing. In the training stage, we build multiple component models to form a multi-branched encoder based on a decision tree (DSDT). The DSDT is built based on prior knowledge of speech and noisy conditions (the speaker, environment, and signal factors are considered in this paper), where each component of the multi-branched encoder performs a particular mapping from noisy to clean speech along the branch in the DSDT. Finally, a decoder is trained on top of the multi-branched encoder. In the testing stage, noisy speech is first processed by each component model. The multiple outputs from these models are then integrated into the decoder to determine the final enhanced speech. Experimental results show that DAEME is superior to several baseline models in terms of objective evaluation metrics, automatic speech recognition results, and quality in subjective human listening tests.

Towards speech enhancement using a variational U-Net architecture

“…Different types of mask-based methods have been used in the literature, such as ideal binary masks (IBM) and ideal ratio masks (IRM) [3]. Auto-encoder based approaches to speech enhancement favor compact features such as Melfrequency power spectra [4] and short term Fourier transform (STFT) spectra computed across short utterances [5,6,7] or a small temporal context [8]. Deep networks predominantly use higher-dimension log-power spectra with a comparably long temporal context in an attempt to learn features best representing clean speech [9,10].…”

Human Laughter Generation using Hybrid Generative Models

“…So far, autoencoders were used in many audio applications as an analysis-synthesis scheme where the input signals dimension is reduced to a latent vector (encoding), and the signal is regenerated from it (decoding). In [18] authors used Denoising AE to reduce noise and enhance the quality of synthesized speech. In addition, deep autoencoder is used to extract significant features from the spectral envelop which improve the text to speech synthesis procedure [19].…”

Estratégias de Combinação de Espectrogramas de Magnitude e de Fase Aplicadas em Sistemas Robustos de Detecção de Palavras-Chave

“…Os sistemas atuais de reconhecimento automático de fala (automatic speech recognition -ASR) têm exibido desempenho satisfatório em cenários acústicos com níveis de ruído controlados, contudo, em ambientes com baixa razão sinalruído (signal-to-noise ratio -SNR), a operação desses sistemas se torna severamente prejudicada [6]. Nesse contexto, apesar de a robustez ao ruído ainda ser um problema crítico em aplicações do mundo real, a maioria dos trabalhos de pesquisa do estado-da-arte em KWS não tem levado em consideração (de forma eficaz) os efeitos do ruído [7], [8].…”

“…Em [8] e [9], são discutidas diversas estratégias de redução de ruído e realce do sinal de fala. Recentemente, com o desenvolvimento das técnicas de aprendizado profundo, grandes avanços vêm sendo alcançados nessas áreas de aplicação.…”