The impact of the Lombard effect on audio and visual speech recognition systems

Marxer, Ricard; Barker, Jon; Alghamdi, Najwa; Maddock, Steve

doi:10.1016/j.specom.2018.04.006

Cited by 23 publications

(37 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This scenario allows to take into account the two factors responsible for the Lombard adaptation: first, speakers tend to regulate their vocal effort based on the auditory feedback, i.e. they involuntarily react to the perceived level of their own speech [17]; secondly, they change their speaking style to communicate better with others [20,21]. The impact of the noise type on the Lombard effect is currently unclear.…”

Section: Audio-visual Corpus and Noise Datamentioning

confidence: 99%

“…2. In the Lombard GRID database, the energy difference between Lombard and neutral utterances is between 3 dB and 13 dB [17]. If we assume that the listener is immersed in SSN at 80 dB SPL, like in the recording conditions of the Lombard GRID database, and that the conversational speech level is between 60 and 70 dB SPL [29,30], the SNR is between -17 dB and 3 dB.…”

Section: Additive Noise Levelsmentioning

confidence: 99%

See 1 more Smart Citation

Effects of Lombard Reflex on the Performance of Deep-learning-based Audio-visual Speech Enhancement Systems

Michelsanti

Sigurðsson

Jensen

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Humans tend to change their way of speaking when they are immersed in a noisy environment, a reflex known as Lombard effect. Current speech enhancement systems based on deep learning do not usually take into account this change in the speaking style, because they are trained with neutral (non-Lombard) speech utterances recorded under quiet conditions to which noise is artificially added. In this paper, we investigate the effects that the Lombard reflex has on the performance of audio-visual speech enhancement systems based on deep learning. The results show that a gap in the performance of as much as approximately 5 dB between the systems trained on neutral speech and the ones trained on Lombard speech exists. This indicates the benefit of taking into account the mismatch between neutral and Lombard speech in the design of audio-visual speech enhancement systems.Index Terms-Audio-visual speech enhancement, deep learning, Lombard effect

show abstract

Section: Audio-visual Corpus and Noise Datamentioning

confidence: 99%

Section: Additive Noise Levelsmentioning

confidence: 99%

Effects of Lombard Reflex on the Performance of Deep-learning-based Audio-visual Speech Enhancement Systems

Michelsanti

Sigurðsson

Jensen

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Dessa forma, a potência do ruído é reduzida, as distorções na fala são minimizadas e as informações de localização da fonte desejada são preservadas. No entanto, como efeito adverso, o MWF distorce as pistas biauriculares do ruído, fazendo com que a localização percebida do ruído seja a mesma da fonte associada à fala [11]. De forma a controlar o efeito indesejado que o MWF gera sobre as pistas de localização do ruído residual, métodos recentes de redução de ruído incorporam, à função custo original do MWF, um termo adicional de penalização sobre as distorções Diego M. do Carmo, Ricardo A. Borsoi e Márcio H. Costa Universidade Federal de Santa Catarina.…”

Section: Introductionunclassified

Proposta de um parâmetro robusto para redução de ruído e preservação espacial em aparelhos auditivos biauriculares

Carmo¹,

Borsoi²,

Costa³

2021

Anais Do XXXIX Simpósio Brasileiro De Telecomunicações E Processamento De Sinais

View full text Add to dashboard Cite

Resumo-O filtro de Wiener multicanal (MWF) tem sido amplamente utilizado para redução de ruído em aparelhos auditivos biauriculares. Neste trabalho é apresentada uma proposta de parâmetro de ponderação para funções custo aumentadas, baseadas no método MWF com preservação da percepção espacial. De forma a demonstrar seu desempenho, é realizada uma análise matemática, suportada por simulações computacionais do método "MWF com preservação da função de transferência interauricular". Os resultados indicam que, em contraposição ao uso de um parâmetro fixo, a estratégia dinâmica proposta diminui a sensibilidade do ponto de operação projetado ao efeito Lombard.Palavras-Chave-Aparelhos auditivos, pistas biauriculares, redução de ruído.

show abstract

“…Most of both research and application areas are, however, related to human (and human-computer) communication, telecommunications, etc. [9], [19], [20]. Especially important are strategies for improving speech comprehensibility in noisy conditions based on various techniques, including speech modeling.…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Lombard Speech Models in the Context of Speech in Noise Enhancement

et al. 2020

View full text Add to dashboard Cite

The Lombard effect is one of the most well-known effects of noise on speech production. Speech with the Lombard effect is more easily recognizable in noisy environments than normal natural speech. Our previous investigations showed that speech synthesis models might retain Lombard-effect characteristics. In this study, we investigate several speech models, such as harmonic, source-filter, and sinusoidal, applied to Lombard speech in the context of speech enhancement. For this purpose, 100 utterances of natural speech, and 100 with the Lombard effect induced are used. The goal of this study is to check to what extent speech utterances based on these models are recognizable and at what SNR (Signal-to-Noise Ratio) level threshold a particular model stops working. For this purpose, the synthesized models and Lombard speech are mixed with babble speech and street noise recordings with different SNRs. The quality of these models is measured, employing objective indicators as well as subjective tests. Since there is no standardized measure to apply to enhanced speech, an objective measure of assessing the speech quality of a model synthesizing Lombard speech characteristics, based on a feature vector, is proposed. Our approach is then compared with the standardized metric used in telecommunications as well as with subjective test results. The experimental investigations show the superiority of the source-filter models applied to synthesize Lombard speech over other models utilized. Also, the measure proposed correlates more closely with the results of the subjective evaluation than the outcomes from the ITU-T P.563 recommendation. This was checked with a ANOVA statistical analysis.

show abstract

The impact of the Lombard effect on audio and visual speech recognition systems

Cited by 23 publications

References 33 publications

Effects of Lombard Reflex on the Performance of Deep-learning-based Audio-visual Speech Enhancement Systems

Effects of Lombard Reflex on the Performance of Deep-learning-based Audio-visual Speech Enhancement Systems

Proposta de um parâmetro robusto para redução de ruído e preservação espacial em aparelhos auditivos biauriculares

Evaluation of Lombard Speech Models in the Context of Speech in Noise Enhancement

Contact Info

Product

Resources

About