Evaluation of Lombard Speech Models in the Context of Speech in Noise Enhancement

Abstract

The Lombard effect is one of the most well-known effects of noise on speech production. Speech with the Lombard effect is more easily recognizable in noisy environments than normal natural speech. Our previous investigations showed that speech synthesis models might retain Lombard-effect characteristics. In this study, we investigate several speech models, such as harmonic, source-filter, and sinusoidal, applied to Lombard speech in the context of speech enhancement. For this purpose, 100 utterances of natural speech, and 100 with the Lombard effect induced are used. The goal of this study is to check to what extent speech utterances based on these models are recognizable and at what SNR (Signal-to-Noise Ratio) level threshold a particular model stops working. For this purpose, the synthesized models and Lombard speech are mixed with babble speech and street noise recordings with different SNRs. The quality of these models is measured, employing objective indicators as well as subjective tests. Since there is no standardized measure to apply to enhanced speech, an objective measure of assessing the speech quality of a model synthesizing Lombard speech characteristics, based on a feature vector, is proposed. Our approach is then compared with the standardized metric used in telecommunications as well as with subjective test results. The experimental investigations show the superiority of the source-filter models applied to synthesize Lombard speech over other models utilized. Also, the measure proposed correlates more closely with the results of the subjective evaluation than the outcomes from the ITU-T P.563 recommendation. This was checked with a ANOVA statistical analysis.

Lombard effect – is it a remedy for speech communication in noisy environments?

“…Since the discovery of LE, this phenomenon has been extensively studied by a wide range of specialists to find solutions to improve the performance of automatic speech recognition systems in noisy environments (Maheswari et al, 2021) or increase speech intelligibility by converting the speaking style from normal to Lombard speech (Li et al, 2020, Kąkol et al, 2020. Also, the basic idea was that LE might be applied to speech synthesizers, allowing them to adapt to noisy conditions , Paul et al, 2020.…”

Investigation of the Lombard effect based on a machine learning approach

“…In contrast, the LE may create problems when detecting speech in noise automatically, but not trained on data related to the LE (Vlaj and Kacic, 2011;Marxer et al, 2018;Korvel et al, 2020;Maheswari et al, 2020). Such hyper-articulation impairs the performance of the speech recognition systems (Maheswari et al, 2020), so it is essential to train them on data, including Lombard-related features.…”

A Novel Method for Intelligibility Assessment of Nonlinearly Processed Speech in Spaces Characterized by Long Reverberation Times

“…However, it should be remembered that the best judge of speech intelligibility is the human ear. Therefore, there are many attempts to find a correlation between objective measurement results and subjective evaluation [5][6][7][8][9]. In general, the STI value can be determined using two ways, i.e., the direct method based on modulated signals or the indirect method based on the impulse response, according to IEC 60268-16 standard [10].…”