Combining non-negative matrix factorization and deep neural networks for speech enhancement and automatic speech recognition

Vu, Thanh Tung; Bigot, Benjamin; Chng, Eng Siong

doi:10.1109/icassp.2016.7471725

Cited by 27 publications

(25 citation statements)

References 13 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the next sections, we apply these three steps to different cost functions c ji . In each case, we specify the probability density function (pdf) involved in (2). We then choose the cost functions c ji with respect to the possible decisions and hypotheses.…”

Section: Joint Detection and Estimation Approach: General Frameworkmentioning

confidence: 99%

Optimal Bayesian Speech Enhancement by Parametric Joint Detection and Estimation

Pastor

Bey

2020

IEEE Access

View full text Add to dashboard Cite

In this paper, we propose a general framework to estimate short-time spectral amplitudes (STSA) of speech signals in noise by joint speech detection and estimation to remove or reduce background noise, without increasing signal distortion. The approach is motivated by the fact that speech signals have sparse time-frequency representations and can reasonably be assumed not to be present in every timefrequency bin of the time-frequency domain. By combining parametric detection and estimation theories, the main idea is to take into consideration speech presence and absence in each time-frequency bin to improve the performance of Bayesian estimators. In this respect, for three Bayesian estimators, optimal Neyman-Pearson detectors are derived to decide on the absence or presence of speech in each given timefrequency bin. Decisions returned by such detectors are then used to improve the initial estimates. The resulting estimations have been assessed in two scenarios, namely, with and without reference noise power spectrum. The objective tests confirm the relevance of these approaches, both in terms of speech quality and intelligibility. INDEX TERMS Unsupervised speech enhancement, parametric method, joint detection and estimation, Bayesian estimator, minimum mean square error (MMSE).

show abstract

Section: Joint Detection and Estimation Approach: General Frameworkmentioning

confidence: 99%

Optimal Bayesian Speech Enhancement by Parametric Joint Detection and Estimation

Pastor

Bey

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The adaptive masking coefficient (a) is derived by the signal-to-noise ratio, which is used to weight IBM and IRM to get the adaptive mask as the training target for DNN through Eq. (16).…”

Section: Deep Neural Network Modelmentioning

confidence: 99%

“…In the same time, Huang et al put forward a joint optimization of masks and deep recurrent neural networks (DRNN) for Monaural Source Separation algorithm [15]. In 2016, Vu et al also presented a speech enhancement algorithm combining non-negative matrix factorization and deep neural networks [16]. These algorithms mentioned above are all to estimate the amplitude spectrum of the target speech.…”

Section: Introductionmentioning

confidence: 99%

Multi-resolution auditory cepstral coefficient and adaptive mask for speech enhancement with deep neural network

Sun

Liu

et al. 2019

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

The performance of the existing speech enhancement algorithms is not ideal in low signal-to-noise ratio (SNR) non-stationary noise environments. In order to resolve this problem, a novel speech enhancement algorithm based on multi-feature and adaptive mask with deep learning is presented in this paper. First, we construct a new feature called multi-resolution auditory cepstral coefficient (MRACC). This feature which is extracted from four cochleagrams of different resolutions can capture the local information and spectrotemporal context and reduce the algorithm complexity. Second, an adaptive mask (AM) which can track noise change for speech enhancement is put forward. The AM can flexibly combine the advantages of an ideal binary mask (IBM) and an ideal ratio mask (IRM) with the change of SNR. Third, a deep neural network (DNN) architecture is used as a nonlinear function to estimate adaptive mask. And the first and second derivatives of MRACC and MRACC are used as the input of the DNN. Finally, the estimated AM is used to weight the noisy speech to achieve enhanced speech. Experimental results show that the proposed algorithm not only further improves speech quality and intelligibility, but also suppresses more noise than the contrast algorithms. In addition, the proposed algorithm has a lower complexity than the contrast algorithms.

show abstract

“…The ambient noise can affect the signal components of speech and worsen the representation of recognition result. To solve the problem of noise, many methods of mitigating the effect of noise on ASR development, have been developed [ 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 ].…”

Section: Introductionmentioning

confidence: 99%

“…To make the ASR system more robust in noisy environments, the methodology of artificial neural networks (ANN), especially deep neural networks (DNN), has been widely utilized in speech enhancement for ASR in recent years [ 16 , 17 , 18 ]. The goal of DNN is to implement complex nonlinear numeric functions, which are used to directly map log-likelihood spectral features of noisy speech into corresponding clean speech.…”

Section: Introductionmentioning

confidence: 99%

Threshold-Based Noise Detection and Reduction for Automatic Speech Recognition System in Human-Robot Interactions

Lee

Wang

Chen

2018

Sensors

View full text Add to dashboard Cite

This work develops a speech recognition system that uses two procedures of proposed noise detection and combined noise reduction. The system can be used in applications that require interactive robots to recognize the contents of speech that includes ambient noise. The system comprises two stages, which are the threshold-based noise detection and the noise reduction procedure. In the first stage, the proposed system automatically determines when to enhance the quality of speech based on the signal-to-noise ratio (SNR) values of the collected speech at all times. In the second stage, independent component analysis (ICA) and subspace speech enhancement (SSE) are employed for noise reduction. Experimental results reveal that the SNR values of the enhanced speech exceed those of the received noisy speech by approximately 20 dB to 25 dB. The noise reduction procedure improves the speech recognition rates by around 15% to 25%. The experimental results indicate that the proposed system can reduce the effect of noise in numerous noisy environments and improve the quality of speech for recognition purposes.

show abstract

Combining non-negative matrix factorization and deep neural networks for speech enhancement and automatic speech recognition

Cited by 27 publications

References 13 publications

Optimal Bayesian Speech Enhancement by Parametric Joint Detection and Estimation

Optimal Bayesian Speech Enhancement by Parametric Joint Detection and Estimation

Multi-resolution auditory cepstral coefficient and adaptive mask for speech enhancement with deep neural network

Threshold-Based Noise Detection and Reduction for Automatic Speech Recognition System in Human-Robot Interactions

Contact Info

Product

Resources

About