A Perceptual Weighting Filter Loss for DNN Training In Speech Enhancement

Zhao, Ziyue; Elshamy, Samy; Fingscheidt, Tim

doi:10.1109/waspaa.2019.8937189

Cited by 21 publications

(26 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As can be observed, all frequency bins have equal importance without any perceptual considerations, such as the masking property of the human ear [29], or the loudness difference [31]. Furthermore, as the MSE loss is optimized in a global fashion, the network may learn to completely attenuate some regions of the noisy spectrum, where the noise component is significantly higher compared to the speech component.…”

Section: Baseline Losses 231 Baseline Msementioning

confidence: 99%

“…The parameters of the deep learning architectures are then optimized by minimizing the MSE between the inferred results and their corresponding targets. In reality, optimization of the MSE loss in training does not guarantee any perceptual quality of the speech component and of the residual noise component, respectively, which leads to limited performance [27][28][29][30][31][32][33][34][35][36]. This effect is even more evident when the level of the noise component is significantly higher than that of the speech component in some regions of the noisy speech spectrum, which explains the bad performance at lower SNR conditions when training with MSE.…”

Section: Introductionmentioning

confidence: 99%

“…To keep more speech component details and to constrain the speech distortion to an acceptable level, Shivakumar et al [27] assigned a high penalty against speech component removal in the conventional MSE loss function during training, which results in an improvement in speech quality metrics. A perceptually weighted loss function that emphasizes important TF regions has recently been proposed in [28,29], improving speech intelligibility.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Components loss for neural networks in mask-based speech enhancement

Elshamy

Zhao

et al. 2021

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

Estimating time-frequency domain masks for single-channel speech enhancement using deep learning methods has recently become a popular research field with promising results. In this paper, we propose a novel components loss (CL) for the training of neural networks for mask-based speech enhancement. During the training process, the proposed CL offers separate control over preservation of the speech component quality, suppression of the noise component, and preservation of a naturally sounding residual noise component. We illustrate the potential of the proposed CL by evaluating a standard convolutional neural network (CNN) for mask-based speech enhancement. The new CL is compared to several baseline losses, comprising the conventional mean squared error (MSE) loss w.r.t. speech spectral amplitudes or w.r.t. an ideal-ratio mask, auditory-related loss functions, such as the perceptual evaluation of speech quality (PESQ) loss and the perceptual weighting filter loss, and also the recently proposed SNR loss with two masks. Detailed analysis suggests that the proposed CL obtains a better or at least a more balanced performance across all employed instrumental quality metrics, including SNR improvement, speech component quality, enhanced total speech quality, and particularly also delivers a natural sounding residual noise component. For unseen noise types, we excel even perceptually motivated losses by an about 0.2 points higher PESQ score. The recently proposed so-called SNR loss with two masks not only requires a network with more parameters due to the two decoder heads, but also falls behind on PESQ and POLQA and particularly w.r.t. residual noise quality. Note that the proposed CL shows significantly more 1st ranks among the evaluation metrics than any other baseline. It is easy to implement, and code is provided at https://github.com/ifnspaml/Components-Loss.

show abstract

Section: Baseline Losses 231 Baseline Msementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Components loss for neural networks in mask-based speech enhancement

Elshamy

Zhao

et al. 2021

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

show abstract

“…To do so, we computed cost functions on outputs at three stages of the training. The first loss is the perceptually weighted error of the log-magnitude spectra, i.e., = ∑ ∑ ‖( ( , ) −̂( , )) + log(| ( , )| + 10 −8 )‖ 1 (1) where ̂( , ) is the 65-dimensional log-magnitude spectra output from DNN1, ( , ) is the log-magnitude spectra of the clean speech, and ( , ) is the perceptual weight function widely adopted in speech codecs [20]. The second loss counts the estimation error of envelopes, i.e.,…”

Section: Neural Network-based Ace (Nnace)mentioning

confidence: 99%

A Noise-Robust Signal Processing Strategy for Cochlear Implants Using Neural Networks

Zheng

Shi

Kang

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Signal processing strategies in most clinical cochlear implants (CIs) extract and transmit speech envelopes to stimulate the auditory neurons. The incomplete representation of the rich fine structures in speech has significantly degraded the CI recipients' ability in highlevel perception, including their speech understanding in noise. This paper presents a noise-robust signal processing strategy to deal with this problem. Neural networks (NN) are built and trained to simulate the advanced combination encoder (ACE, a strategy for CI products of Cochlear Corporation). The NN-based ACE (namely, NNACE) is trained with a sophisticatedly designed loss function to output envelope-like signals that 1) is compatible with ACE-based CI system and can serve as the modulator to generate the electric stimuli, 2) is more noise-robust, and 3) might bear a certain degree of the temporal fine structures of speech. Subjective and objective evaluations with vocoder simulated speech show that NNACE outperforms the other methods and further actual CI experiments are warranted.

show abstract

“…A masking‐based weighting is used to mask noise at frequencies where speech energy is dominant [28]. Perceptual weighting filter loss that places less emphasis near formant peaks and more emphasis on spectral valleys is shown to have the ability to improve perceptual quality [29, 30]. Some commonly used metrics in SE are also modified and designed as loss functions to improve its performance.…”

Section: Introductionmentioning

confidence: 99%

Improving speech enhancement by focusing on smaller values using relative loss

Deng

et al. 2020

IET signal process.

View full text Add to dashboard Cite

The task of single-channel speech enhancement is to restore clean speech from noisy speech. Recently, speech enhancement has been greatly improved with the introduction of deep learning. Previous work proved that using ideal ratio mask or phase-sensitive mask as intermediation to recover clean speech can yield better performance. In this case, the mean square error is usually selected as the loss function. However, after conducting experiments, the authors find that the mean square error has a problem. It considers absolute error values, meaning that the gradients of the network depend on absolute differences between estimated values and true values, so the points in magnitude spectra with smaller values contribute little to the gradients. To solve this problem, they propose relative loss, which pays more attention to relative differences between magnitude spectra, rather than the absolute differences, and is more in accordance with human sensory characteristics. The perceptual evaluation of speech quality, the short-time objective intelligibility, the signal-to-distortion ratio, and the segmental signal-to-noise ratio are used to evaluate the performance of the relative loss. Experimental results show that it can greatly improve speech enhancement by focusing on smaller values.

show abstract

A Perceptual Weighting Filter Loss for DNN Training In Speech Enhancement

Cited by 21 publications

References 24 publications

Components loss for neural networks in mask-based speech enhancement

Components loss for neural networks in mask-based speech enhancement

A Noise-Robust Signal Processing Strategy for Cochlear Implants Using Neural Networks

Improving speech enhancement by focusing on smaller values using relative loss

Contact Info

Product

Resources

About