OOV Recovery with Efficient 2nd Pass Decoding and Open-vocabulary Word-level RNNLM Rescoring for Hybrid ASR

Zhang, Xiaohui; Povey, Daniel; Khudanpur, Sanjeev

doi:10.1109/icassp40776.2020.9053872

Cited by 4 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Decoding involves finding the most probable path of word sequences. A Weighted Finite State Transducer (WFST) (13) is used for this purpose. It is represented as given in Equation (2).…”

Section: Asr Decodingmentioning

confidence: 99%

“…Decoding generates lattices. A lattice contains the paths with higher scores and is derived from the pruned subset of the decoding graph (13) . Lattice rescoring is a process where the path probabilities are updated with the new language model probabilities while retaining the transition and pronunciation probabilities.…”

Section: Lattice Rescoringmentioning

confidence: 99%

“…In this case, the unknown words are replaced with subsequences to aid better recognition. The most recent work is by Zhang et al [50] where a hybrid lexical model with phonemes and words is used to generate OOV candidates with phoneme constraints. Instead of hybrid subword models, parallel models also have been used for OOV detection and recovery where both subsequence lattices and word lattices are used to determine OOV [45,51,52,53,54].…”

Section: Oov Detection and Recoverymentioning

confidence: 99%

“…OOV recovery follows OOV detection. P2G mapping is the most popular for OOV recovery where the subsequences are stitched together to form words using P2G models [55,50]. Another approach is clustering or similarity-based approach where words similar to reference or in-vocabulary words to determine OOV probabilities [56,57,58,59,60].…”

Section: Oov Detection and Recoverymentioning

confidence: 99%

See 3 more Smart Citations

Low Resource Kannada Speech Recognition using Lattice Rescoring and Speech Synthesis

Murthy¹,

Sitaram²

2023

IJST

View full text Add to dashboard Cite

Objectives: Improving the accuracy of low resource speech recognition in a model trained on only 4 hours of transcribed continuous speech in Kannada language, using data augmentation. Methods: Baseline language model is augmented with unigram counts of words, that are present in the Wikipedia text corpus but absent in the baseline, for initial decoding. Lattice rescoring is then applied using the language model augmented with Wikipedia text. Speech synthesis-based augmentation with multi-speaker syllable-based synthesis, using voices in Kannada and cross-lingual Telugu languages, is employed. We synthesize basic syllables, syllables with consonant conjuncts, and words that contain syllables that are absent in the training speech, for Kannada language. Findings: An overall word error rate (WER) of 9.04% is achieved over a baseline WER of 40.93%. Language model augmentation and lattice rescoring gives an absolute improvement of 16.68%. Applying our method of syllable-based speech synthesis over language model augmentation and rescoring yields a total reduction of 31.89% in WER. The proposed approach of language model augmentation is memory efficient and consumes only 1/8 th the memory required for decoding with Wikipedia augmented language model (2 gigabytes versus 18 gigabytes) while giving comparable WER (22.95% for Wikipedia versus 24.25% for our method). Augmentation with synthesized syllables enhances the ability of the speech recognition model to recognize basic sounds thus improving recognition of out-of-vocabulary words to 90% and in-vocabulary words to 97%. Novelty: We propose novel methods of language model augmentation and synthesis-based augmentation to achieve low WER for a speech recognition model trained on only 4 hours of continuous speech. Obtaining high recognition accuracy (or low WER) for very small speech corpus is a challenge. In this paper, we demonstrate that high accuracy can be achieved using data augmentation for a small corpus-based speech recognition.

show abstract

“…Decoding involves finding the most probable path of word sequences. A Weighted Finite State Transducer (WFST) (13) is used for this purpose. It is represented as given in Equation (2).…”

Section: Asr Decodingmentioning

confidence: 99%

Section: Lattice Rescoringmentioning

confidence: 99%