SNR-Based Progressive Learning of Deep Neural Network for Speech Enhancement

Gao, Tian; Du, Jun; Dai, Li-Rong; Lee, Chin‐Hui

doi:10.21437/interspeech.2016-224

Cited by 77 publications

(45 citation statements)

References 29 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the generalization ability in mismatched conditions is the main problem of deep learning based method. Inspired by our previous work [21,22], we adopt an advanced LSTM architecture with the novel design of hidden layers via densely connected progressive learning and output layer via multiple-target learning. The overall LSTM architecture aims to predict the clean LPS features given the input noisy log-power spectra (LPS) features with acoustic context.…”

Section: Speech Denoisingmentioning

confidence: 99%

“…This stacking style network can learn multiple targets progressively and efficiently. In order to make full use of the rich set of information from the multiple learning targets, we update the progressive learning in [22] with dense structures [23] in which the input and the estimations of intermediate target are spliced together to learn next target. Then, a weighted MMSE criterion in terms of multitask learning (MTL) is designed to optimize all network parameters randomly initialized with K target layers as follows:…”

Section: Speech Denoisingmentioning

confidence: 99%

See 1 more Smart Citation

Speaker Diarization with Enhancing Speech for the First DIHARD Challenge

Sun

Jiang³

et al. 2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

We design a novel speaker diarization system for the first DI-HARD challenge by integrating several important modules of speech denoising, speech activity detection (SAD), i-vector design, and scoring strategy. One main contribution is the proposed long short-term memory (LSTM) based speech denoising model. By fully utilizing the diversified simulated training data and advanced network architecture using progressive multitask learning with dense structure, the denoising model demonstrates the strong generalization capability to realistic noisy environments. The enhanced speech can boost the performance for the subsequent SAD, segmentation and clustering. To the best of our knowledge, this is the first time we show significant improvements of deep learning based single-channel speech enhancement over state-of-the-art diarization systems in highly mismatch conditions. For the design of i-vector extraction, we adopt a residual convolutional neural network trained on large dataset including more than 30,000 people. Finally, by score fusion of different i-vectors based on all these techniques, our systems yield diarization error rates (DERs) of 24.56% and 36.05% on the evaluation sets of Track1 and Track2, which are both in the second place among 14 and 11 participating teams, respectively.

show abstract

Section: Speech Denoisingmentioning

confidence: 99%

Section: Speech Denoisingmentioning

confidence: 99%

Speaker Diarization with Enhancing Speech for the First DIHARD Challenge

Sun

Jiang³

et al. 2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…To improve generalization, a common method is to increase the size of the network, such as adding more hidden layers for progressive training [23], using multistage networks [24] or adopting end-to-end approaches. Santiago Pascual et al propose a SEGAN to learn complex functions from large example sets [25].…”

Section: In the 1980s Ephraim And Malah Proposed The Minimummentioning

confidence: 99%

Real-Time Speech Enhancement Algorithm Based on Attention LSTM

et al. 2020

View full text Add to dashboard Cite

Because traditional single-channel speech enhancement algorithms are sensitive to the environment and perform poorly, a speech enhancement algorithm based on attention-gated long short-term memory (LSTM) is proposed. To simulate human auditory perceptual characteristics, the algorithm divides the frequency band according to the Bark scale. Based on these bands, bark frequency cepstral coefficients (BFCCs), their derivative features and pitch-based features are extracted. Furthermore, considering that different noises have different influence on the clean speech, the attention mechanism is applied to screen out the information less polluted by noise, which is helpful to reconstruct the clean speech. To adaptively reallocate the power ratio of the speech and noise during the construction of the ratio mask, the ideal ratio mask (IRM) with the inter-channel correlation (ICC) is adopted as the learning target. In addition, to improve the performance of the network, the algorithm introduces a multiobjective learning strategy to jointly optimize the networks by using a voice activity detector (VAD). Subjective and objective experiments show that the proposed algorithm outperforms other baseline algorithms. In real-time experiment, the proposed algorithm maintains high real-time performance and fast convergence speed. INDEX TERMS Speech enhancement, long short-term memory, attention mechanism, bark scale.

show abstract

“…In recent years, deep neural network (DNN)-based speech enhancement methods have shown significant performance advantages over the traditional approaches in complex noise environments, even the extremely nonstationary noises. Whether utilizing masking-based [9][10][11] or mapping-based [12][13][14][15][16][17] DNN methods, their general rule is to optimize the loss function between the ideal and noisy targets to achieve as little error as possible in the global noisy speech dataset. Consequently, richer datasets, and a better objective function and neural network models were further explored to guarantee the robust generalization ability of DNN models to cope with the diversified noise environments in real life.…”

Section: Introductionmentioning

confidence: 99%

Environmental Attention-Guided Branchy Neural Network for Speech Enhancement

et al. 2020

View full text Add to dashboard Cite

The performance of speech enhancement algorithms can be further improved by considering the application scenarios of speech products. In this paper, we propose an attention-based branchy neural network framework by incorporating the prior environmental information for noise reduction. In the whole denoising framework, first, an environment classification network is trained to distinguish the noise type of each noisy speech frame. Guided by this classification network, the denoising network gradually learns respective noise reduction abilities in different branches. Unlike most deep neural network (DNN)-based methods, which learn speech reconstruction capabilities with a common neural structure from all training noises, the proposed branchy model obtains greater performance benefits from the specially trained branches of prior known noise interference types. Experimental results show that the proposed branchy DNN model not only preserved better enhanced speech quality and intelligibility in seen noisy environments, but also obtained good generalization in unseen noisy environments.for the performance improvement of DNN-based speech denoising. Generative adversarial networks (GAN) [22] have been another new solution to train the DNN model to learn noise suppression abilities. These methods aim to train a common DNN denoising model for all kinds of noise interference in different application scenarios.However, most speech products have their specific application scenarios. When the application scenario is determined, the types of noise interference are known. In different noise environments, the optimal parameters of speech enhancement algorithms are different. As investigated in [23], this prior environmental information was helpful to further improve the speech enhancement methods to achieve better noise suppression effect in specific noise environments. Therefore, some researchers started to design the speech enhancement algorithms by incorporating the prior noise information. In [24][25][26], the noise classification module was integrated into the Wiener filter and some statistical model-based speech estimators. Guided by the noise classification module for optimal parameter selection, the performance of traditional methods was further improved in prior known noisy environments. A similar idea was applied to the DNN-based denoising algorithms [27][28][29]. In the recent studies of [28,29], under the guidance of the noise classification unit, several independent DNN models were trained to give full play to their respective noise reduction capabilities in different noise environments. Although this "divide and conquer" strategy effectively improves the noise reduction performance, it also increases the storage burden and reduces the complementarity between different noises.In this work, we proposed a novel branchy neural network (BNN) framework to improve the storage burden and noise complementarity problem of separate training. There are two key modules working together in our proposed framework, a classification...

show abstract

SNR-Based Progressive Learning of Deep Neural Network for Speech Enhancement

Cited by 77 publications

References 29 publications

Speaker Diarization with Enhancing Speech for the First DIHARD Challenge

Speaker Diarization with Enhancing Speech for the First DIHARD Challenge

Real-Time Speech Enhancement Algorithm Based on Attention LSTM

Environmental Attention-Guided Branchy Neural Network for Speech Enhancement

Contact Info

Product

Resources

About