Hierarchical statistical language models: experiments on in-domain adaptation

Abstract

We introduce a hierarchical statistical language model, represented as a collection of local models plus a general sentence model. We provide an example that mixes a trigram general model and a PFSA local model for the class of decimal numbers, described in terms of sub-word units (graphemes). This model practically extends the vocabulary of the overall model to an infinite size, but still has better performance compared to a word-based model.Using in-domain language model adaptation experiments, we show that local models can encode enough linguistic information, if well trained, that they may be ported to new language models without re-estimation.

Evaluating hierarchical hybrid statistical language models

“…Due to lack of space, we only highlight some important aspects that we want to improve on. For further details on the model, we refer to [3].…”

“…The rationale is that different parts of a sentence could be described more finely by such local models than by a conventional n-gram model trained on a large corpus. For further benefits of this approach, including portability of local models, we refer to the companion paper in these proceedings, [3].…”