Improving language modeling by using distance and co-occurrence information of word-pairs and its application to LVCSR

Chong, Tze Yuang; Banchs, Rafael E.; Chng, Eng Siong; Li, Haizhou

doi:10.1109/icassp.2014.6854530

Cited by 1 publication

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we will highlight the advantages of the proposed TD and TO model components in exploiting the distant context to improve the -gram model. Also, as an extension to the previous works [25], [26], we will further show the capability of the proposed approach for easing the scarcity problem of conventional -gram modeling, in particular, for higher orders of -grams. In addition, we will show the benefits of the proposed approach in different applications, besides speech recognition which was first discussed in [26].…”

Section: Introductionmentioning

confidence: 96%

See 1 more Smart Citation

Decoupling Word-Pair Distance and Co-occurrence Information for Effective Long History Context Language Modeling

Chong

Banchs

Chng

et al. 2015

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

In this paper, we propose the use of distance and co-occurrence information of word-pairs to improve language modeling. We have empirically shown that, for history-context sizes of up to ten words, the extracted information about distance and co-occurrence complements the -gram language model well, for which learning long-history contexts is inherently difficult. Evaluated on the Wall Street Journal and the Switchboard corpora, our proposed model reduces the trigram model perplexity by up to 11.2% and 6.5%, respectively. As compared to the distant bigram model and the trigger model, our proposed model offers a more effective manner of capturing far context information, as verified in terms of perplexity and computational efficiency, i.e., fewer free parameters to be fine-tuned. Experiments using the proposed model for speech recognition, text classification and word prediction tasks showed improved performance.

show abstract

Section: Introductionmentioning

confidence: 96%

“…Inspired by the two ideas above, a novel approach to -gram probability estimation has been recently proposed by Chong et al [25], [26]. In this new approach, distance and co-occurrence information within word-pairs are decoupled and modeled independently of each other.…”

Section: Introductionmentioning

confidence: 99%