Query language modeling for voice search

Abstract

The paper presents an empirical exploration of google.com query stream language modeling. We describe the normalization of the typed query stream resulting in out-of-vocabulary (OoV) rates below 1% for a one million word vocabulary. We present a comprehensive set of experiments that guided the design decisions for a voice search service. In the process we re-discovered a less known interaction between Kneser-Ney smoothing and entropy pruning, and found empirical evidence that hints at non-stationarity of the query stream, as well as strong dependence on various English locales-USA, Britain and Australia.

Distributed discriminative language models for Google voice-search

“…We use maximum likelihood (ML) trained single mixture Gaussians for our weakAM. We use a baseline LM that is sufficiently small (21 million n-grams) to allow for sub-real time lattice generation on the training data with a small memory footprint, without compromising on its strength: as shown in [12], it takes much larger LMs to get a significant relative gain in WER.…”

Speech and Multimodal Interaction in Mobile Search

“…For example, Google's English voice search model was trained from a 230 billion word corpus with 1 million unique words [9]. There are a few directions that researchers and engineers have been recently exploring to improve the quality of the LM for voice search applications.…”

Investigation of Combining Various Major Language Model Technologies including Data Expansion and Adaptation

“…The first one is the scarcity of training data. For instance, in the voice search task, it is easy to collect data corresponding to the target task by accessing text-input query logs [8]. In the spontaneous speech task, on the other hand, the data corresponding to the target task must be obtained by manually transcribing speech.…”