Local Sparsity Based Online Dictionary Learning for Environment-Adaptive Speech Enhancement with Nonnegative Matrix Factorization

Abstract

In this paper, a nonnegative matrix factorization (NMF)-based speech enhancement method robust to real and diverse noise is proposed by online NMF dictionary learning without relying on prior knowledge of noise. Conventional NMF-based methods have used a fixed noise dictionary, which often results in performance degradation when the NMF noise dictionary cannot cover noise types that occur in real-life recording. Thus, the noise dictionary needs to be learned from noises according to the variation of recording environments. To this end, the proposed method first estimates noise spectra and then performs online noise dictionary learning by a discriminative NMF learning framework. In particular, the noise spectra are estimated from minimum mean squared error filtering, which is based on the local sparsity defined by a posteriori signal-to-noise ratio (SNR) estimated from the NMF separation of the previous analysis frame. The effectiveness of the proposed speech enhancement method is demonstrated by adding six different realistic noises to clean speech signals with various SNRs. Consequently, it is shown that the proposed method outperforms comparative methods in terms of signal-to-distortion ratio (SDR) and perceptual evaluation of speech quality (PESQ) for all kinds of simulated noise and SNR conditions.

Convolutional Recurrent Neural Network-Based Event Detection in Tunnels Using Multiple Microphones

“…Thus, instead of directly using trueD^i;cfalse^i, an additional filtering process is designed here to take into account such noise variation. Similar to [38], a minimum mean squared error (MMSE) filter is constructed to obtain noise components for online noise learning.…”

“…Next, M frames of trueD˜i;cfalse^i are also concatenated as trueD˜i = [trueD˜i−M+1;cfalse^i,⋯,trueD˜i;cfalse^i] to apply a discriminative dictionary learning technique [38] such as:trueB˜l,iD=trueB˜l−1,iD ∘(trueD˜itrueB˜l−1,iD trueA^L,iD)(…”

Multi-Task Learning U-Net for Single-Channel Speech Enhancement and Mask-Based Voice Activity Detection

“…This section first evaluates the performance of the proposed MTU-Net-based SE method and then compares it with those of several conventional SE methods based on SNMF [3], SEGAN [8], DRNN [5], and U-Net [15]. Here, SNMF, DRNN, SEGAN, and U-Net were trained with hyperparameters according to [3,5,8,15], respectively. In particular, the proposed MTU-Net-based SE method was implemented in three different ways: MTU-Net(spec), MTU-Net(IRM), and MTU-Net(IBM).…”

“…It attempts to remove background noise from a noisy signal using a single microphone or a microphone array. There have been many studies on statistical SE techniques, including Wiener filtering, the minimum mean square error (MMSE)-based spectral amplitude estimator [1], and non-negative matrix factorization (NMF) [2,3]. Among them, sparse NMF (SNMF) achieves the best performance in noise reduction with matched noise that is modeled by the noise basis.…”

“…Finally, the estimated IRM is applied to noisy input spectra to produce a clean speech spectrum estimate, which is more accurate than that obtained from the conventional U-Net. The speech quality and intelligibility of the proposed MTU-Net-based SE method are compared to those of statistical SE methods, especially SNMF [3], and several deep learning-based SE methods, including the conventional U-Net [15], SEGAN [8], and DRNN [5]. In addition to estimating the clean speech spectrum, the estimated IRM can be used to detect voiced intervals.…”