Voice search language model adaptation using contextual information

Scheiner, Justin; Williams, Ian; Aleksic, Petar S.

doi:10.1109/slt.2016.7846273

Cited by 16 publications

(25 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many speech recognition applications where on-thefly adjustment is indispensable. In the voice search task, we have previously shown quality improvements by introducing ngram weight adjustments for salient n-grams from personal contexts and geographic information [2] as well as improved recognition of contact names using context [11]. Products such as the Google Assistant use context for all types of personal entities (e.g.…”

Section: Contextual Speech Recognitionmentioning

confidence: 99%

“…Contextual signals can include: a user's location, the device being used, or personalization information such as a user's favorite songs and calendar events ( Figure 1). Including this information has been shown to improve recognition results [2]. Our contextual ASR system was previously built on a conventional architecture, and in this paper we propose a design to bring similar improvements to E2E architectures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Contextual Speech Recognition in End-to-end Neural Network Systems Using Beam Search

Williams¹,

Kannan²,

Aleksic³

et al. 2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

Recent work has shown that end-to-end (E2E) speech recognition architectures such as Listen Attend and Spell (LAS) can achieve state-of-the-art quality results in LVCSR tasks. One benefit of this architecture is that it does not require a separately trained pronunciation model, language model, and acoustic model. However, this property also introduces a drawback: it is not possible to adjust language model contributions separately from the system as a whole. As a result, inclusion of dynamic, contextual information (such as nearby restaurants or upcoming events) into recognition requires a different approach from what has been applied in conventional systems. We introduce a technique to adapt the inference process to take advantage of contextual signals by adjusting the output likelihoods of the neural network at each step in the beam search. We apply the proposed method to a LAS E2E model and show its effectiveness in experiments on a voice search task with both artificial and real contextual information. Given optimal context, our system reduces WER from 9.2% to 3.8%. The results show that this technique is effective at incorporating context into the prediction of an E2E system.

show abstract

Section: Contextual Speech Recognitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Contextual Speech Recognition in End-to-end Neural Network Systems Using Beam Search

Williams¹,

Kannan²,

Aleksic³

et al. 2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our approach is inspired by the empirical observations that (1) NEs often occur in common word contexts that are easily recognized by a first-pass decoder (e.g., "call X mobile"); (2) when an NE is misrecognized, the incorrect hypothesis still approximates the ground-truth phonemes spoken by the user. For example, the utterance "call Goudzwaard" may be misrecognized as "call god's word", but the presence of "call" indicates a possible NE, while the phonetics are sufficiently close to match the missing NE based on knowledge of user state (e.g., the name "Goudzwaard" is in the user's contact list).…”

Section: System Designmentioning

confidence: 99%

“…Many techniques have been developed to take advantage of contextual signals. For example, in language model (LM)based ASR systems, rescoring methods are described that dynamically adjust LM weights; some reweigh n-grams appearing in the user's context on the fly [1,2,3], while others dynamically expand the LM via class grammars [4]. Contextual ASR (biasing) methods for end-to-end systems have also been proposed [5,6].…”

Section: Introductionmentioning

confidence: 99%

Contextual Recovery of Out-of-Lattice Named Entities in Automatic Speech Recognition

et al. 2019

Self Cite

View full text Add to dashboard Cite

As voice-driven intelligent assistants become commonplace, adaptation to user context becomes critical for Automatic Speech Recognition (ASR) systems. For example, ASR systems may be expected to recognize a user's contact names containing improbable or out-of-vocabulary (OOV) words. We introduce a method to identify contextual cues in a firstpass ASR system's output and to recover out-of-lattice hypotheses that are contextually relevant. Our proposed module is agnostic to the architecture of the underlying recognizer, provided it generates a word lattice of hypotheses; it is sufficiently compact for use on device. The module identifies subgraphs in the lattice likely to contain named entities (NEs), recovers phoneme hypotheses over corresponding time spans, and inserts NEs that are phonetically close to those hypotheses. We measure a decrease in the mean word error rate (WER) of word lattices from 11.5% to 4.9% on a test set of NEs.

show abstract

“…Rescoring adjusts LM probabilities in real time based on contextual signals, and allows targeted adjustments without the need for training a contextspecific LM. Work has been done on incorporating a variety of contextual signals into ASR: the device type being used, the history of the speaker's queries or actions, device state, location and dialog state, among others [1,2,3].…”

Section: Introductionmentioning

confidence: 99%

Semantic Lattice Processing in Contextual Automatic Speech Recognition for Google Assistant

et al. 2018

Self Cite

View full text Add to dashboard Cite

Recent interest in intelligent assistants has increased demand for Automatic Speech Recognition (ASR) systems that can utilize contextual information to adapt to the user's preferences or the current device state. For example, a user might be more likely to refer to their favorite songs when giving a "music playing" command or request to watch a movie starring a particular favorite actor when giving a "movie playing" command. Similarly, when a device is in a "music playing" state, a user is more likely to give volume control commands. In this paper, we explore using semantic information inside the ASR word lattice by employing Named Entity Recognition (NER) to identify and boost contextually relevant paths in order to improve speech recognition accuracy. We use broad semantic classes comprising millions of entities, such as songs and musical artists, to tag relevant semantic entities in the lattice. We show that our method reduces Word Error Rate (WER) by 12.0% relative on a Google Assistant "media playing" commands test set, while not affecting WER on a test set containing commands unrelated to media.

show abstract

Voice search language model adaptation using contextual information

Cited by 16 publications

References 9 publications

Contextual Speech Recognition in End-to-end Neural Network Systems Using Beam Search

Contextual Speech Recognition in End-to-end Neural Network Systems Using Beam Search

Contextual Recovery of Out-of-Lattice Named Entities in Automatic Speech Recognition

Semantic Lattice Processing in Contextual Automatic Speech Recognition for Google Assistant

Contact Info

Product

Resources

About