An Equal Data Setting for Attention-Based Encoder-Decoder and HMM/DNN Models: A Case Study in Finnish ASR

Rouhe, Aku; Camp, Astrid Van; Singh, Mittul; hamme, Hugo Van; Kurimo, Mikko

doi:10.1007/978-3-030-87802-3_54

Cited by 6 publications

(12 citation statements)

References 25 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compared the AED models against HMM systems in an equal data setting (Rouhe et al, 2021 ): both paradigms only used transcribed speech as the training data. Since HMM systems typically leverage additional text data and expert lexica, comparing them with end-to-end models only trained on transcribed speech confounds differences in models and learning with differences in the training data.…”

Section: Modelsmentioning

confidence: 99%

Finnish parliament ASR corpus

Virkkunen

Rouhe

Phan

et al. 2023

Lang Resources & Evaluation

Self Cite

View full text Add to dashboard Cite

Public sources like parliament meeting recordings and transcripts provide ever-growing material for the training and evaluation of automatic speech recognition (ASR) systems. In this paper, we publish and analyse the Finnish Parliament ASR Corpus, the most extensive publicly available collection of manually transcribed speech data for Finnish with over 3000 h of speech and 449 speakers for which it provides rich demographic metadata. This corpus builds on earlier initial work, and as a result the corpus has a natural split into two training subsets from two periods of time. Similarly, there are two official, corrected test sets covering different times, setting an ASR task with longitudinal distribution-shift characteristics. An official development set is also provided. We developed a complete Kaldi-based data preparation pipeline and ASR recipes for hidden Markov models (HMM), hybrid deep neural networks (HMM-DNN), and attention-based encoder-decoders (AED). For HMM-DNN systems, we provide results with time-delay neural networks (TDNN) as well as state-of-the-art wav2vec 2.0 pretrained acoustic models. We set benchmarks on the official test sets and multiple other recently used test sets. Both temporal corpus subsets are already large, and we observe that beyond their scale, HMM-TDNN ASR performance on the official test sets has reached a plateau. In contrast, other domains and larger wav2vec 2.0 models benefit from added data. The HMM-DNN and AED approaches are compared in a carefully matched equal data setting, with the HMM-DNN system consistently performing better. Finally, the variation of the ASR accuracy is compared between the speaker categories available in the parliament metadata to detect potential biases based on factors such as gender, age, and education.

show abstract

Section: Modelsmentioning

confidence: 99%

Finnish parliament ASR corpus

Virkkunen

Rouhe

Phan

et al. 2023

Lang Resources & Evaluation

Self Cite

View full text Add to dashboard Cite

show abstract

“…For improved speech recognition accuracy, DNN-HMM methods are best suited for languages with limited annotated speech [14]. Also, when much more text data is available than speech data, DNN-HMM models are the preferred choice [15,16] than the modern E2E approaches. Additionally, DNN-HMM ASR models offer the advantage of easy integration into small hardware devices, enabling fast on-device speech recognition [14].…”

Section: Baby Elephant Compound Word Formed By Agglutination Of Nouns...mentioning

confidence: 99%

Improving speech recognition systems for the morphologically complex Malayalam language using subword tokens for language modeling

Manohar,

A R,

Rajan

2023

J AUDIO SPEECH MUSIC PROC.

View full text Add to dashboard Cite

This article presents the research work on improving speech recognition systems for the morphologically complex Malayalam language using subword tokens for language modeling. The speech recognition system is built using a deep neural network–hidden Markov model (DNN-HMM)-based automatic speech recognition (ASR). We propose a novel method, syllable-byte pair encoding (S-BPE), that combines linguistically informed syllable tokenization with the data-driven tokenization method of byte pair encoding (BPE). The proposed method ensures words are always segmented at valid pronunciation boundaries. On a text corpus that has been divided into tokens using the proposed method, we construct statistical n-gram language models and assess the modeling effectiveness in terms of both information-theoretic and corpus linguistic metrics. A comparative study of the proposed method with other data-driven (BPE, Morfessor, and Unigram), linguistic (Syllable), and baseline (Word) tokenization algorithms is also presented. Pronunciation lexicons of subword tokenized units are built with pronunciation described as graphemes. We develop ASR systems employing the subword tokenized language models and pronunciation lexicons. The resulting ASR models are comprehensively evaluated to answer the research questions regarding the impact of subword tokenization algorithms on language modeling complexity and on ASR performance. Our study highlights the strong performance of the hybrid S-BPE tokens, achieving a notable 10.6% word error rate (WER), which represents a substantial 16.8% improvement over the baseline word-level ASR system. The ablation study has revealed that the performance of S-BPE segmentation, which initially underperformed compared to syllable tokens with lower amounts of textual data for language modeling, exhibited steady improvement with the increase in LM training data. The extensive ablation study indicates that there is a limited advantage in raising the n-gram order of the language model beyond $$n=3$$ n = 3 . Such an increase results in considerable model size growth without significant improvements in WER. The implementation of the algorithm and all associated experiments are available under an open license, allowing for reproduction, adaptation, and reuse.

show abstract

“…This has two benefits. Firstly, the end-to-end AED models and the HMM/DNN system can then be compared in an equal data setting [14]. Pure end-to-end models are only trained on transcribed speech.…”

Section: Language Modeling Datamentioning

confidence: 99%

“…We use grapheme-based lexica and SentencePiece Byte Pair Encoding subword units [21] in language modeling. The lexicon transducer requires special word-position dependent phone handling [14,22]. The language models are built with VariKN [23], which allows growing the modified Kneser-Ney models up to 10-gram scale.…”

Section: Hmm System Language Modelsmentioning

confidence: 99%

Low Resource Comparison of Attention-based and Hybrid ASR Exploiting wav2vec 2.0

Rouhe¹,

Virkkunen²,

Leinonen³

et al. 2022

Interspeech 2022

View full text Add to dashboard Cite

Low resource speech recognition can potentially benefit a lot from exploiting a pretrained model such as wav2vec 2.0. These pretrained models have learned useful representations in an unsupervised or self-supervised task, often leveraging a very large corpus of untranscribed speech. The pretrained models can then be used in various ways. In this work we compare two approaches which exploit wav2vec 2.0: an attention-based end-to-end model (AED), where the wav2vec 2.0 model is used in the model encoder, and a hybrid hidden Markov model (HMM/DNN) speech recognition system, where the wav2vec 2.0 model is used in the acoustic model. These approaches are compared in a very difficult Northern Sámi task, as well as an easier, simulated low resource task in Finnish. We find that the wav2vec 2.0 AED models can learn a working attention mechanism, but are still outperformed by wav2vec 2.0 HMM/DNN systems. Our best wav2vec 2.0 HMM/DNN recipe on 20 hours is competitive with an HMM/DNN system trained on 1600 hours.

show abstract

An Equal Data Setting for Attention-Based Encoder-Decoder and HMM/DNN Models: A Case Study in Finnish ASR

Cited by 6 publications

References 25 publications

Finnish parliament ASR corpus

Finnish parliament ASR corpus

Improving speech recognition systems for the morphologically complex Malayalam language using subword tokens for language modeling

Low Resource Comparison of Attention-based and Hybrid ASR Exploiting wav2vec 2.0

Contact Info

Product

Resources

About