Supervised Single Channel Speech Enhancement Based on Dual-Tree Complex Wavelet Transforms and Nonnegative Matrix Factorization Using the Joint Learning Process and Subband Smooth Ratio Mask

Abstract

In this paper, we propose a novel speech enhancement method based on dual-tree complex wavelet transforms (DTCWT) and nonnegative matrix factorization (NMF) that exploits the subband smooth ratio mask (ssRM) through a joint learning process. The discrete wavelet packet transform (DWPT) suffers the absence of shift invariance, due to downsampling after the filtering process, resulting in a reconstructed signal with significant noise. The redundant stationary wavelet transform (SWT) can solve this shift invariance problem. In this respect, we use efficient DTCWT with a shift invariance property and limited redundancy and calculate the ratio masks (RMs) between the clean training speech and noisy speech (i.e., training noise mixed with clean speech). We also compute RMs between the noise and noisy speech and then learn both RMs with their corresponding clean training clean speech and noise. The auto-regressive moving average (ARMA) filtering process is applied before NMF in previously generated matrices for smooth decomposition. An ssRM is proposed to exploit the advantage of the joint use of the standard ratio mask (sRM) and square root ratio mask (srRM). In short, the DTCWT produces a set of subband signals employing the time-domain signal. Subsequently, the framing scheme is applied to each subband signal to form matrices and calculates the RMs before concatenation with the previously generated matrices. The ARMA filter is implemented in the nonnegative matrix, which is formed by considering the absolute value. Through ssRM, speech components are detected using NMF in each newly formed matrix. Finally, the enhanced speech signal is obtained via the inverse DTCWT (IDTCWT). The performances are evaluated by considering an IEEE corpus, the GRID audio-visual corpus, and different types of noises. The proposed approach significantly improves objective speech quality and intelligibility and outperforms the conventional STFT-NMF, DWPT-NMF, and DNN-IRM methods.

Single-channel Speech Separation Based on Double-density Dual-tree CWT and SNMF

“…The magnitude spectrum 𝐙𝐌 𝐛,𝐭𝐥 is not equal to the sum of the preliminary approximation 𝐗𝐌 𝐚,𝐭𝐥 and 𝐘𝐌 𝐚,𝐭𝐥 . To avoid errors, we compute the SBRM using Equation (12) and Equation (13).…”

“…Furthermore, STFT suffers from this time-frequency difficulty due to a lack of knowledge about which frequency exists at which moments. Recently, wavelet-based separation methods [12][13][14] have emerged for researchers to overcome the abovementioned problems. In [12], the discrete wavelet transform (DWT) divides a signal into lowfrequency approximation and high-frequency details coefficients.…”

“…Recently, wavelet-based separation methods [12][13][14] have emerged for researchers to overcome the abovementioned problems. In [12], the discrete wavelet transform (DWT) divides a signal into lowfrequency approximation and high-frequency details coefficients. This method has reduced separation time, but the separated sign seriously affects individual speakers' intelligibility.…”

A Smart Binaural Hearing Aid Architecture Based on a Mobile Computing Platform

“…The noise reduction of binaural HA has been proven to be more effective than two HA systems, which process noise independently, because the former can use more spatial information [15]. However, the binaural interconnection is the biggest challenge impeding the development of binaural HA [16][17][18][19][20]. In spite of the advances in HA technology, balancing the algorithms and user experience remains a challenge, mainly due to the limitations of current HA structures.…”