Unsupervised Equalization of Lombard Effect for Speech Recognition in Noisy Adverse Environments

Abstract

Abstract-In the presence of environmental noise, speakers tend to adjust their speech production in an effort to preserve intelligible communication. The noise-induced speech adjustments, called Lombard effect (LE), are known to severely impact the accuracy of automatic speech recognition (ASR) systems. The reduced performance results from the mismatch between the ASR acoustic models trained typically on noise-clean neutral (modal) speech and the actual parameters of noisy LE speech. In this study, novel unsupervised frequency domain and cepstral domain equalizations that increase ASR resistance to LE are proposed and incorporated in a recognition scheme employing a codebook of noisy acoustic models. In the frequency domain, short-time speech spectra are transformed towards neutral ASR acoustic models in a maximum-likelihood fashion. Simultaneously, dynamics of cepstral samples are determined from the quantile estimates and normalized to a constant range. A codebook decoding strategy is applied to determine the noisy models best matching the actual mixture of speech and noisy background. The proposed algorithms are evaluated side by side with conventional compensation schemes on connected Czech digits presented in various levels of background car noise. The resulting system provides an absolute word error rate (WER) reduction on 10-dB signal-to-noise ratio data of 8.7% and 37.7% for female neutral and LE speech, respectively, and of 8.7% and 32.8% for male neutral and LE speech, respectively, when compared to the baseline recognizer employing perceptual linear prediction (PLP) coefficients and cepstral mean and variance normalization.Index Terms-Cepstral compensation, codebook of noisy models, frequency warping, Lombard effect, speech recognition.

Masking release for hearing-impaired listeners: The effect of increased audibility through reduction of amplitude variability

“…Relevant examples include level and/or frequency equalization techniques which attempt to transform the speech-plus-noise signal so that its features mimic a set of reference features calculated in the temporal, spectral, or cepstral domains. One line of work equalizes the noisy input signal to reflect the characteristics of the clean speech used to train the ASR system (Hilger and Ney, 2006;Joshi et al, 2011), while other work has focused on undoing the characteristics of Lombard speech (Boril and Hansen, 2010). Other techniques involve more complex models of intelligibility with the explicit goal of enhancing some intelligibility metric and may operate on clean speech prior to the addition of noise (Chanda and Park, 2007).…”

UT-Vocal Effort II: Analysis and constrained-lexicon recognition of whispered speech

“…Previous studies on robust ASR for stressed speech and Lombard effect speech have reported performance gains when altering configurations of the front-end feature extraction filter banks [26][27][28]. Inspired by [28], our first step is to replace the Mel and Bark filter banks (FB) in MFCC and PLP by a bank of triangular and rectangular filters uniformly distributed over a linear frequency axis.…”

“…Inspired by [28], our first step is to replace the Mel and Bark filter banks (FB) in MFCC and PLP by a bank of triangular and rectangular filters uniformly distributed over a linear frequency axis. In the case of the triangular bank, the band cutoffs are located at the center frequencies of the adjacent filters while the rectangular filters are stacked next to each other without overlap as in [27].…”

Engineering Analysis and Recognition of Nigerian English: An Insight into Low Resource Languages

“…The fundamental frequency of speech (F 0 ) is known to be affected by stress [19,20], emotions [19,21], and talking styles [22]. Different languages may exhibit unique F 0 characteristics [23] and the same may be observed also for individual dialects of a language [24].…”