LIUM and CRIM ASR System Combination for the REPERE Evaluation Campaign

Lectures are usually known to be highly specialised in that they deal with multiple and domain specific topics. This context is challenging for Automatic Speech Recognition (ASR) systems since they are sensitive to topic variability. Language Model (LM) adaptation is a commonly used technique to address the mismatch problem between training and test data. In this paper, we are interested in a qualitative analysis in order to relevantly compare the accuracy of the LM adaptation. While word error rate is the most common metric used to evaluate ASR systems, we consider that this metric cannot provide accurate information. Consequently, we explore the use of other metrics based on individual word error rate, indexability, and capability of building relevant requests for information retrieval from the ASR outputs. Experiments are carried out on the PASTEL corpus, a new dataset in French language, composed of lecture recordings, manual chaptering, manual transcriptions, and slides. While an adapted LM allows us to reduce the global classical word error rate by 15.62% in relative, we show that this reduction reaches 44.2% when computed on relevant words only. These observations are confirmed with the high LM adaptation gains obtained with indexability and information retrieval metrics.

show abstract

“…The vocabulary of the generic language model contains around 160k words. More details about language models can be found in [18].…”

Section: Language Model Adaptation and Asr Descriptionmentioning

confidence: 99%

Qualitative Evaluation of ASR Adaptation in a Lecture Context: Application to the PASTEL Corpus

Mdhaffar¹,

Estève²,

Hernández³

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…For these experiments, a variant of the ASR system developed by LIUM that won the last evaluation campaign on French language has been used [23]. This system is based on the Kaldi speech recognition toolkit [24].…”

Section: Lium Asr System Dedicated To Mediamentioning

confidence: 99%

ASR Error Management for Improving Spoken Language Understanding

Simonnet¹,

Ghannay²,

Camelin³

et al. 2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

This paper addresses the problem of automatic speech recognition (ASR) error detection and their use for improving spoken language understanding (SLU) systems. In this study, the SLU task consists in automatically extracting, from ASR transcriptions, semantic concepts and concept/values pairs in a e.g touristic information system. An approach is proposed for enriching the set of semantic labels with error specific labels and by using a recently proposed neural approach based on word embeddings to compute well calibrated ASR confidence measures. Experimental results are reported showing that it is possible to decrease significantly the Concept/Value Error Rate with a state of the art system, outperforming previously published results performance on the same experimental data. It also shown that combining an SLU approach based on conditional random fields with a neural encoder/decoder attention based architecture, it is possible to effectively identifying confidence islands and uncertain semantic output segments useful for deciding appropriate error handling actions by the dialogue manager strategy.

show abstract

“…For this experiment, we use the automatic transcriptions of the French MEDIA corpus [27,28] generated by a variant of the ASR system developed by LIUM that won the last evaluation campaign on French language [29]. This variant system contains 2.5K words and its language model is adapted to the MEDIA data.…”

Section: Spoken Language Understanding Taskmentioning

confidence: 99%

Enriching Confusion Networks for Post-processing

Ghannay

Estève

Camelin

2017

Statistical Language and Speech Processing

Self Cite

View full text Add to dashboard Cite

The paper proposes a new approach for a posteriori enrichment of automatic speech recognition (ASR) confusion networks (CNs). CNs are usually needed to decrease word error rate and to compute confidence measures, but they are also used in many ways in order to improve post-processing of ASR outputs. For instance, they can be helpfully used to propose alternative word hypotheses when ASR outputs are corrected by a human on post-edition. However, CNs bins do not have a fixed length, and sometimes contain only one or two word hypotheses: in this case the number of alternatives to correct a misrecognized word is very low, reducing the chance of helping the human annotator. Our approach for CN enrichment is based on a new similarity measure presented in this paper, computed from acoustic and linguistic word embeddings, that allows us to take into consideration both acoustic and linguistic similarities between two words. Experimental results show that our approach is relevant: enriched CNs (for a bin size equals to 6) increase the potential correction of erroneous words by 23% than initial CNs produced by an ASR system. In our experiments, a spoken language understanding task is also targeted.

show abstract

LIUM and CRIM ASR System Combination for the REPERE Evaluation Campaign

Cited by 30 publications

References 11 publications

Qualitative Evaluation of ASR Adaptation in a Lecture Context: Application to the PASTEL Corpus

Qualitative Evaluation of ASR Adaptation in a Lecture Context: Application to the PASTEL Corpus

ASR Error Management for Improving Spoken Language Understanding

Enriching Confusion Networks for Post-processing

Contact Info

Product

Resources

About