Snr-Based Teachers-Student Technique For Speech Enhancement

Hao, Xiang; Su, Xiangdong; Wang, Zhiyu; Zhang, Qiang; Xu, Huali; Gao, Guanglai

doi:10.1109/icme46284.2020.9102846

Cited by 10 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hinton et al [16] trained a smaller student network with labels generated by a stronger teacher model, comprising an ensemble of models. Hao et al [17] used an SNR-based TS method to train an SE model using an ensemble of different models, which were individually trained for different SNR Ranges. Kobayashi et al [18] used KD to train a unidirectional recurrent student network with a bi-directional teacher network inspired by Born Again Networks [19].…”

Section: Related Workmentioning

confidence: 99%

Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation

Thakker¹,

Eskimez²,

Yoshioka³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper investigates how to improve the runtime speed of personalized speech enhancement (PSE) networks while maintaining the model quality. Our approach includes two aspects: architecture and knowledge distillation (KD). We propose an end-to-end enhancement (E3Net) model architecture, which is 3× faster than a baseline STFT-based model. Besides, we use KD techniques to develop compressed student models without significantly degrading quality. In addition, we investigate using noisy data without reference clean signals for training the student models, where we combine KD with multi-task learning (MTL) using an automatic speech recognition (ASR) loss. Our results show that E3Net provides better speech and transcription quality with a lower target speaker over-suppression (TSOS) rate than the baseline model. Furthermore, we show that the KD methods can yield student models that are 2 − 4× faster than the teacher and provides reasonable quality. Combining KD and MTL improves the ASR and TSOS metrics without degrading the speech quality.

show abstract

Section: Related Workmentioning

confidence: 99%

Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation

Thakker¹,

Eskimez²,

Yoshioka³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…[22] designed a two-stage training distillation method and a co-worker-based network to improve the performance of SE. In the time domain, to improve performance at both low and high signal-to-noise ratios (SNRs), [23] built multiple teachers trained under SNRs and then transferred knowledge to the student network. Further, [24] applied standard KD [10] to reduce the system latency while preventing performance degradation.…”

Section: Introductionmentioning

confidence: 99%

Multi-View Attention Transfer for Efficient Speech Enhancement

Shin¹,

Park²,

Kim³

et al. 2022

Interspeech 2022

View full text Add to dashboard Cite

Recent deep learning models have achieved high performance in speech enhancement; however, it is still challenging to obtain a fast and low-complexity model without significant performance degradation. Previous knowledge distillation studies on speech enhancement could not solve this problem because their output distillation methods do not fit the speech enhancement task in some aspects. In this study, we propose multi-view attention transfer (MV-AT), a feature-based distillation, to obtain efficient speech enhancement models in the time domain. Based on the multi-view features extraction model, MV-AT transfers multi-view knowledge of the teacher network to the student network without additional parameters. The experimental results show that the proposed method consistently improved the performance of student models of various sizes on the Valentini and deep noise suppression (DNS) datasets. MANNER-S-8.1GF with our proposed method, a lightweight model for efficient deployment, achieved 15.4× and 4.71× fewer parameters and floating-point operations (FLOPs), respectively, compared to the baseline model with similar performance.

show abstract

“…First, for challenging acoustic scenarios as low SNR conditions, current SE systems usually suffer from performance bottlenecks in recovering clean speech from mixtures (Li et al 2021). Second, noise intensities in real-world scenes change dynamically, which requires SE systems to accommodate wide expansion of the SNR range and raises the difficulty of the network design (Hao et al 2020). The third problem is caused by the limited kernel size of convolution layers, which often results in a short-sighted feature extractor.…”

Section: Introductionmentioning

confidence: 99%

A Repetitive Spectrum Learning Framework for Monaural Speech Enhancement in Extremely Low SNR Environments (Student Abstract)

Tai

2022

AAAI

View full text Add to dashboard Cite

Monaural speech enhancement (SE) at an extremely low signal-to-noise ratio (SNR) condition is a challenging problem and rarely investigated in previous studies. Most SE methods experience failures in this situation due to three major factors: overwhelmed vocals, expanded SNR range, and short-sighted feature processing modules. In this paper, we present a novel and general training paradigm dubbed repetitive learning (RL). Unlike curriculum learning that focuses on learning multiple different tasks sequentially, RL is more inclined to learn the same content repeatedly where the knowledge acquired in previous stages can be used to facilitate calibrating feature representations. We further propose an RL-based end-to-end SE method named SERL. Experimental results on TIMIT dataset validate the superior performance of our method.

show abstract

Snr-Based Teachers-Student Technique For Speech Enhancement

Cited by 10 publications

References 5 publications

Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation

Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation

Multi-View Attention Transfer for Efficient Speech Enhancement

A Repetitive Spectrum Learning Framework for Monaural Speech Enhancement in Extremely Low SNR Environments (Student Abstract)

Contact Info

Product

Resources

About