Transforming Binary Uncertainties for Robust Speech Recognition

Abstract

Abstract-Recently, several algorithms have been proposed to enhance noisy speech by estimating a binary mask that can be used to select those time-frequency regions of a noisy speech signal that contain more speech energy than noise energy. This binary mask encodes the uncertainty associated with enhanced speech in the linear spectral domain. The use of the cepstral transformation smears the information from the noise dominant time-frequency regions across all the cepstral features. We propose a supervised approach using regression trees to learn the nonlinear transformation of the uncertainty from the linear spectral domain to the cepstral domain. This uncertainty is used by a decoder that exploits the variance associated with the enhanced cepstral features to improve robust speech recognition. Systematic evaluations on a subset of the Aurora4 task using the estimated uncertainty show substantial improvement over the baseline performance across various noise conditions. Index Terms-Binary time-frequency mask, computational auditory scene analysis (CASA), robust automatic speech recognition, spectrogram reconstruction, uncertainty decoding.

Coupling binary masking and robust ASR

“…We also performed reconstruction using a method described in [10] using a 1024-component diagonal Gaussian mixture model of clean speech. It was noted in our earlier work that this reconstruction method does not outperform direct masking when a bottom-up mask is used.…”

“…Such methods can be broadly categorized into three groups: 1) extracting robust features like PLP, RASTA [1], and AFE [2], 2) model adaptation techniques like MLLR [3], PMC [4], and Vector Taylor series (VTS) based adaptation [5], and 3) noise suppression or feature enhancement techniques like Wiener filtering [6], VTS-based enhancement [7], and model based feature enhancement [8]. There are also systems that combine the above methods [9,10]. Because of the huge variability in noise in real-life conditions, the level of robustness obtained by these methods is still inadequate.…”

Robust speaker identification using auditory features and computational auditory scene analysis

“…Specifically, we employ a CASA system [24] to segregate speech from noise and obtain a binary mask that indicates reliable or corrupted components of an auditory feature. The auditory feature is then enhanced by reconstructing the corrupted components [18,24,25].…”

“…Additionally, we estimate reconstruction uncertainties [24,25] and apply them in an uncertainty decoder [6] to calculate speaker likelihoods. This decoder accounts for varied accuracies of the feature enhancement process.…”

“…To enhance a corrupted GF, we reconstruct its missing components from a speech prior [18]. Specifically, the missing components are estimated as the expected value conditioned on the reliable data [4,24,25]. Reconstruction errors are estimated as GF uncertainties [24,25].…”