The ESAT 2008 system for N-Best Dutch speech recognition benchmark

Abstract

This paper describes the ESAT 2008 Broadcast News transcription system for the N-Best 2008 benchmark, developed in part for testing the recent SPRAAK Speech Recognition Toolkit. ESAT system was developed for the Southern Dutch Broadcast News subtask of N-Best using standard methods of modern speech recognition. A combination of improvements were made in commonly overlooked areas such as text normalization, pronunciation modeling, lexicon selection and morphological modeling, virtually solving the out-of-vocabulary (OOV) problem for Dutch by reducing OOV-rate to 0.06% on the N-Best development data and 0.23% on the evaluation data. Recognition experiments were run with several configurations comparing one-pass vs. two-pass decoding, high-order vs. low-order ngram models, lexicon sizes and different types of morphological modeling. The system achieved 7.23% word error rate (WER) on the broadcast news development data and 20.3% on the much more difficult evaluation data of N-Best.

N-Best 2008: A Benchmark Evaluation for Large Vocabulary Speech Recognition in Dutch

“…This includes the scores and recognition hypothesis files of the best scoring system, allowing future researchers to compare the output of their own recognition systems for Dutch to the state-of the art of 2008. The evaluation has generated at least five papers in conference proceedings [1,3,9,11,22]. It remains to be seen if follow-on evaluations of Dutch speech recognition sparks enough enthusiasm from sponsors and participating developers to be realised.…”

“…Dutch is a morphological language with strong compounding (similar to German) and is moderately inflectional. This requires large vocabularies [1,3] of 300-500k words, but this is not uncommon for languages like German. ESAT reported language models based on morphological analysis [1], which resulted in a moderate reduction in out-of-vocabulary rate and WER during development for smaller vocabulary sizes.…”

“…This requires large vocabularies [1,3] of 300-500k words, but this is not uncommon for languages like German. ESAT reported language models based on morphological analysis [1], which resulted in a moderate reduction in out-of-vocabulary rate and WER during development for smaller vocabulary sizes. Vecsys Research C Limsi [3] used a common pronunciation dictionary for Northern and Southern Dutch, which was further adapted to the task by including pronunciation frequencies obtained from forced-aligned training data.…”

“…During the preparations of the evaluation [21], it was decided that in written publications comparative results [22] are to be presented anonymously, but that individual sites can of course present their own results [1,3,9]. This was inspired by the way it goes in the very successful NIST Speaker Recognition campaigns, and the most important reason for N-Best was to make the evaluation more attractive for industrial participants.…”

Improving n-gram probability estimates by compound-head clustering

“…Vocabularies of V words always contain the V most frequent words in Mediargus. They were converted into phonemic lexica using an updated version of [16] and integrated, together with the created LMs (see Section 5.3), into the recognizer described in [17]. The semantic head mapper is the same that was used in [14], except for the adaptations mentioned in the previous sections.…”