A Comparison of Perceptually Motivated Loss Functions for Binary Mask Estimation in Speech Separation

Abstract

This work proposes and compares perceptually motivated loss functions for deep learning based binary mask estimation for speech separation. Previous loss functions have focused on maximising classification accuracy of mask estimation but we now propose loss functions that aim to maximise the hit minus false-alarm (HIT-FA) rate which is known to correlate more closely to speech intelligibility. The baseline loss function is binary cross-entropy (CE), a standard loss function used in binary mask estimation, which maximises classification accuracy. We propose first a loss function that maximises the HIT-FA rate instead of classification accuracy. We then propose a second loss function that is a hybrid between CE and HIT-FA, providing a balance between classification accuracy and HIT-FA rate. Evaluations of the perceptually motivated loss functions with the GRID database show improvements to HIT-FA rate and ESTOI across babble and factory noises. Further tests then explore application of the perceptually motivated loss functions to a larger vocabulary dataset.

Speech Segregation in Background Noise Based on Deep Learning

“…Bregman in his book attributed auditory segregation to Auditory Scene Analysis (ASA) and summarized the segregation process into two stages: segmentation and grouping [6]. ASA procedure was applied by the individual acoustic system to segregate sound [7]. ASA evaluated and recuperated single and distinct sound from an amalgam of noises to generate expressive speech essentials once the sound fundamentals had been removed.…”

DNN Driven Speaker Independent Audio-Visual Mask Estimation for Speech Separation

“…In the literature, extensive research has been carried out to develop speech separation methods for speech recognition [6,7]. Researchers have proposed several different speech separation models such as parametric mask estimation methods [8,9,10], neural network based mask estimation methods [11,12], and novel loss functions [13]. However, limited work has been conducted to develop robust speaker independent audio-visual speech separation models to perform enhancement.…”

“…The few attempts to address this problem have been restricted to speaker dependent scenarios. In [13], audio and visual features are first concatenated into a single vector. The concatenated vector is then used to train a non-causal speakerdependent DNN with a perceptually motivated loss function inspired by the hit minus false-alarm (HIT-FA) rate.…”

A hybrid technique for speech segregation and classification using a sophisticated deep neural network

“…The human auditory system segregates sound using a process known as auditory scene analysis (ASA) [ 10 ]. ASA analyses and recovers single and individual sound from a mixture of sounds to produce meaningful speech elements after removing noise elements.…”