Class phrase models for language modelling

Ries, Klaus; Buo, F.D.; Waibel, Alex

doi:10.21437/icslp.1996-77

Cited by 7 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to similar approaches in which alternative modelling units to words have been investigated [3,5] the method adopted ¥ E.W.D.Whittaker was supported in this research by an EPSRC studentship. His current affiliation is with Compaq Computer Corporation's Cambridge Research Laboratory (edw@crl.dec.com).…”

Section: The Particle ¤ -Gram Modelmentioning

confidence: 99%

Particle-based language modelling

Whittaker¹,

Woodland²

2000

6th International Conference on Spoken Language Processing (ICSLP 2000)

View full text Add to dashboard Cite

This paper investigates the use of particle (sub-word) £ -grams for language modelling. One linguistics-based and two datadriven algorithms are presented and evaluated in terms of perplexity for Russian and English. Interpolating word trigram and particle 6-gram models gives up to a 7.5% perplexity reduction over the baseline word trigram model for Russian. Lattice rescoring experiments are also performed on 1997 DARPA Hub4 evaluation lattices where the interpolated model gives a 0.4% absolute reduction in word error rate over the baseline word trigram model.

show abstract

Section: The Particle ¤ -Gram Modelmentioning

confidence: 99%

Particle-based language modelling

Whittaker¹,

Woodland²

2000

6th International Conference on Spoken Language Processing (ICSLP 2000)

View full text Add to dashboard Cite

show abstract

“…The process is repeated until perplexity stop decreasing. Ries also uses perplexity as an optimisation criterion [3]. The only difference with Giachin is that Ries extracts, at each iteration, a set of candidate word sequences and integrates into its vocabulary only those sequences that reduce perplexity.…”

Section: Principal Variable-length Sequence Modelsmentioning

confidence: 99%

Beyond the conventional statistical language models: the variable-length sequences approach

Zitouni¹,

Smaïli²,

Haton³

2000

6th International Conference on Spoken Language Processing (ICSLP 2000)

View full text Add to dashboard Cite

In natural language, several sequences of words are very frequent. A classical language model, like n-gram, does not adequately take into account such sequences, because it underestimates their probabilities. A better approach consists in modelling word sequences as if they were individual dictionary elements. Sequences are considered as additional entries of the word lexicon, on which language models are computed. In this paper, we present an original method for automatically determining the most important phrases in corpora. This method is based on information theoretic criteria, which insure a high statistical consistency, and on French grammatical classes which include additional type of linguistic dependencies. In addition, the perplexity is used in order to make the decision of selecting a potential sequence more accurate. We propose also several variants of language models with and without word sequences. Among them, we present a model in which the trigger pairs are more significant linguistically. The originality of this model, compared with the commonly used trigger approaches, is the use of word sequences to estimate the trigger pair without limiting itself to single words. Experimental tests, in terms of perplexity and recognition rate, are carried out on a vocabulary of 20000 words and a corpus of 43 million words. The use of word sequences proposed by our algorithm reduces perplexity by more than 16% compared to those, which are limited to single words. The introduction of these word sequences in our dictation machine improves the accuracy by approximately 15%.

show abstract