Domain-Aware Neural Language Models for Speech Recognition

Liu, Linda; Gu, Yile; Gourav, Aditya; Gandhe, Ankur; Kalmane, Shashank; Filimonov, Denis; Rastrow, Ariya; Bulyko, Ivan

doi:10.1109/icassp39728.2021.9414800

Cited by 14 publications

(3 citation statements)

References 22 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To select the rescoring LM, we use the domain aware rescoring framework described in [22] to differentiate between the utterances with contact names and generic ones. For the generic utterances, we use an NCE based neural LM (NLM) [23] trained on 80 million utterances from live traffic.…”

Section: Second-pass Rescoringmentioning

confidence: 99%

Personalization Strategies for End-to-End Speech Recognition Systems

Gourav

Liu

Gandhe

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

The recognition of personalized content, such as contact names, remains a challenging problem for end-to-end speech recognition systems. In this work, we demonstrate how first-and second-pass rescoring strategies can be leveraged together to improve the recognition of such words. Following previous work, we use a shallow fusion approach to bias towards recognition of personalized content in the first-pass decoding. We show that such an approach can improve personalized content recognition by up to 16% with minimum degradation on the general use case. We describe a fast and scalable algorithm that enables our biasing models to remain at the wordlevel, while applying the biasing at the subword level. This has the advantage of not requiring the biasing models to be dependent on any subword symbol table. We also describe a novel second-pass de-biasing approach: used in conjunction with a first-pass shallow fusion that optimizes on oracle WER, we can achieve an additional 14% improvement on personalized content recognition, and even improve accuracy for the general use case by up to 2.5%.

show abstract

Section: Second-pass Rescoringmentioning

confidence: 99%

Personalization Strategies for End-to-End Speech Recognition Systems

Gourav

Liu

Gandhe

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Domain classifiers and other auxiliary models can be used to improve generalization properties. This, however, results in a more complex second pass architecture and higher maintenance costs [16]. Including textual context directly in a neural model has been shown effective in many tasks such as document classification [17], language modeling [18], and acoustic modeling [19].…”

Section: Introductionmentioning

confidence: 99%

Personalization for BERT-based Discriminative Speech Recognition Rescoring

Kolehmainen¹,

Gu²,

Gourav³

et al. 2023

Interspeech 2023

View full text Add to dashboard Cite

Recognition of personalized content remains a challenge in end-to-end speech recognition. We explore three novel approaches that use personalized content in a neural rescoring step to improve recognition: gazetteers, prompting, and a crossattention based encoder-decoder model. We use internal deidentified en-US data from interactions with a virtual voice assistant supplemented with personalized named entities to compare these approaches. On a test set with personalized named entities, we show that each of these approaches improves word error rate by over 10%, against a neural rescoring baseline. We also show that on this test set, natural language prompts can improve word error rate by 7% without any training and with a marginal loss in generalization. Overall, gazetteers were found to perform the best with a 10% improvement in word error rate (WER), while also improving WER on a general test set by 1%.

show abstract

“…Various methods have been proposed to mitigate this issue, which include using mixture of domain experts [4], context based interpolation weights [5] and second-pass rescoring through domain-adapted models [6] to feature based domain adaptation [7]. In [8,9], user-provided speech patterns were leveraged for on-the-fly adaptation.…”

Section: Introductionmentioning

confidence: 99%

Remember the context! ASR slot error correction through memorization

Bekal¹,

Shenoy²,

Sunkara³

et al. 2021

Preprint

View full text Add to dashboard Cite

Accurate recognition of slot values such as domain specific words or named entities by automatic speech recognition (ASR) systems forms the core of the Goal-oriented Dialogue Systems. Although it is a critical step with direct impact on downstream tasks such as language understanding, many domain agnostic ASR systems tend to perform poorly on domain specific or long tail words. They are often supplemented with slot error correcting systems but it is often hard for any neural model to directly output such rare entity words. To address this problem, we propose k-nearest neighbor (k-NN) search that outputs domain-specific entities from an explicit datastore. We improve error correction rate by conveniently augmenting a pretrained joint phoneme and text based transformer sequence to sequence model with k-NN search during inference. We evaluate our proposed approach on five different domains containing long tail slot entities such as full names, airports, street names, cities, states. Our best performing error correction model shows a relative improvement of 7.4% in word error rate (WER) on rare word entities over the baseline and also achieves a relative WER improvement of 9.8% on an out of vocabulary (OOV) test set.

show abstract

Domain-Aware Neural Language Models for Speech Recognition

Cited by 14 publications

References 22 publications

Personalization Strategies for End-to-End Speech Recognition Systems

Personalization Strategies for End-to-End Speech Recognition Systems

Personalization for BERT-based Discriminative Speech Recognition Rescoring

Remember the context! ASR slot error correction through memorization

Contact Info

Product

Resources

About