Speaker-Aware Deep Denoising Autoencoder with Embedded Speaker Identity for Speech Enhancement

Chuang, Fu-Kai; Wang, Syu-Siang; Hung, Jeih-weih; Tsao, Yu; Fang, Shih-Hau

doi:10.21437/interspeech.2019-2108

Cited by 25 publications

(11 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [77], a speaker-aware SE system was proposed and shown to provide improved performance. In this work, we conducted an additional set of experiments that integrate the speaker-aware techniques into the DAEME (termed DAEME-UAT (SA) ).…”

Section: Experiments On the Tmhint Datasetmentioning

confidence: 99%

“…In this work, we conducted an additional set of experiments that integrate the speaker-aware techniques into the DAEME (termed DAEME-UAT (SA) ). For DAEME-UAT (SA) , we included the speaker information in the decoder used on the same approach proposed in [77]. More specifically, we first extracted embedded speaker identity features using a pre-trained DNN model, which was trained to classify a frame-wise speech feature into a certain speaker identity.…”

Section: Experiments On the Tmhint Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Speech Enhancement Based on Denoising Autoencoder With Multi-Branched Encoders

Zezario

Wang

et al. 2020

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Deep learning-based models have greatly advanced the performance of speech enhancement (SE) systems. However, two problems remain unsolved, which are closely related to model generalizability to noisy conditions: (1) mismatched noisy condition during testing, i.e., the performance is generally suboptimal when models are tested with unseen noise types that are not involved in the training data; (2) local focus on specific noisy conditions, i.e., models trained using multiple types of noises cannot optimally remove a specific noise type even though the noise type has been involved in the training data. These problems are common in real applications. In this paper, we propose a novel denoising autoencoder with a multi-branched encoder (termed DAEME) model to deal with these two problems. In the DAEME model, two stages are involved: training and testing. In the training stage, we build multiple component models to form a multi-branched encoder based on a decision tree (DSDT). The DSDT is built based on prior knowledge of speech and noisy conditions (the speaker, environment, and signal factors are considered in this paper), where each component of the multi-branched encoder performs a particular mapping from noisy to clean speech along the branch in the DSDT. Finally, a decoder is trained on top of the multi-branched encoder. In the testing stage, noisy speech is first processed by each component model. The multiple outputs from these models are then integrated into the decoder to determine the final enhanced speech. Experimental results show that DAEME is superior to several baseline models in terms of objective evaluation metrics, automatic speech recognition results, and quality in subjective human listening tests.

show abstract

Section: Experiments On the Tmhint Datasetmentioning

confidence: 99%

Section: Experiments On the Tmhint Datasetmentioning

confidence: 99%

Speech Enhancement Based on Denoising Autoencoder With Multi-Branched Encoders

Zezario

Wang

et al. 2020

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Bhat et al [1] enhanced dysarthric speech features to match that of healthy control speech by a time-delay neural network-based DDA. Chuang et al [14] integrate a DDA with an embedding speaker identity for speech enhancement.…”

Section: Introductionmentioning

confidence: 99%

Multi-Task Transformer with Input Feature Reconstruction for Dysarthric Speech Recognition

Ding

Sun

Zhao

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Dysarthria is a motor speech disorder caused by damage to the part of the nervous system that controls the physical production of speech. It poses great challenges in building robust dysarthric speech recognition (DSR) due to the high inter-and intra-speaker variability. To this end, we propose a multi-task Transformer with input feature reconstruction as an auxiliary task, where the main task of DSR and the auxiliary reconstruction task share the same encoder network. The auxiliary task aims to reconstruct clear speech features from corrupted speech of healthy speakers (intra-domain) or dysarthric speakers (cross-domain). Further, to alleviate the imbalanced distribution of dysarthria data sets, we devise an adaptive rebalance sampling scheme to improve the utterance sampling frequency of dysarthric speech. Experimental results show that the proposed model considerably outperforms other baselines across speakers with varying severity of dysarthria.

show abstract

“…However, the corresponding noise segment in the same environment need to be prepared to create the noise embedding through an embedding subnetwork and the process of speech enhancement during inference is not convenient. Some research related to speaker-aware [13,14] or signal-to-noise-ratio (SNR) aware [15] algorithms have also been proposed to improve the speech enhancement model's denoising performance.…”

Section: Introductionmentioning

confidence: 99%

Neural Noise Embedding for End-To-End Speech Enhancement with Conditional Layer Normalization

Zhang

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Most of the deep learning based speech enhancement methods focus on the modeling of complicated relationship between the noisy speech and the clean speech without the consideration of noise information. In order to cope with various complex noise scenes, we introduce a novel enhancement architecture that integrates a deep autoencoder with neural noise embedding. In this study, a new normalization method, termed conditional layer normalization (CLN), is introduced to improve the generalization of deep learning based speech enhancement approaches for unseen environments. The noise embedding is passed through the CLN layers to regularize the network for speech enhancement task. The proposed network can be adaptively adjusted according to different noise information extracted from the noisy speech input. The network in overall is trained in an end-to-end manner and the experimental results show that the proposed scheme produces satisfactory enhancement performance comparing the other methods. The visualization shows that our proposed network captures noise information, which is helpful to improve robustness to unseen environments for speech enhancement.

show abstract

Speaker-Aware Deep Denoising Autoencoder with Embedded Speaker Identity for Speech Enhancement

Cited by 25 publications

References 31 publications

Speech Enhancement Based on Denoising Autoencoder With Multi-Branched Encoders

Speech Enhancement Based on Denoising Autoencoder With Multi-Branched Encoders

Multi-Task Transformer with Input Feature Reconstruction for Dysarthric Speech Recognition

Neural Noise Embedding for End-To-End Speech Enhancement with Conditional Layer Normalization

Contact Info

Product

Resources

About