A Term Association Translation Model for Naive Bayes Text Classification

Wu, Meng-Sung; Wang, Hsin‐Min

doi:10.1007/978-3-642-30217-6_21

Cited by 3 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such results show that the word-pair distance information provided by the TD model can be used as a more discriminating attribute than the -gram model, which combines both distance and co-occurrences but in a more scarce sensitive way. Although the TO model component did not perform well, several works have stressed the significance of using word association in text classification tasks [42], [43], [44]. These works utilized various methods, including human annotations and some similarity measures, to exploit word association information in the classification task.…”

Section: B Text Classificationmentioning

confidence: 98%

Decoupling Word-Pair Distance and Co-occurrence Information for Effective Long History Context Language Modeling

Chong

Banchs

Chng

et al. 2015

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

In this paper, we propose the use of distance and co-occurrence information of word-pairs to improve language modeling. We have empirically shown that, for history-context sizes of up to ten words, the extracted information about distance and co-occurrence complements the -gram language model well, for which learning long-history contexts is inherently difficult. Evaluated on the Wall Street Journal and the Switchboard corpora, our proposed model reduces the trigram model perplexity by up to 11.2% and 6.5%, respectively. As compared to the distant bigram model and the trigger model, our proposed model offers a more effective manner of capturing far context information, as verified in terms of perplexity and computational efficiency, i.e., fewer free parameters to be fine-tuned. Experiments using the proposed model for speech recognition, text classification and word prediction tasks showed improved performance.

show abstract

Section: B Text Classificationmentioning

confidence: 98%

Decoupling Word-Pair Distance and Co-occurrence Information for Effective Long History Context Language Modeling

Chong

Banchs

Chng

et al. 2015

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Although the TO model component did not perform well in both classification tasks, several works have stressed the significance of using word association in text classification tasks [Antonie and Zaiane, 2002, Cao et al, 2005, Wu and Wang, 2012. These works utilized various methods, including human annotations and some similarity measures, to exploit word association information in the classification task.…”

Section: Application 2: Text Classificationmentioning

confidence: 99%

Exploiting long context using joint distance and occurrence information for language modeling

Chong¹

View full text Add to dashboard Cite

This thesis investigates an approach to exploiting the long context based on the information about the distance and occurrence. By modeling the joint event of distance and occurrence, this approach attempts to incorporate the inter-dependencies into the model, such that information captured from the long context can be more optimally made use.This thesis addresses the problem with the conventional language modeling approaches that tend to neglect the inter-dependencies.Based on the proposed approach, a novel language model, referred to as the termdistance term-occurrence (TDTO) model, is formulated. The TDTO model estimates probabilities based on the events of term-distance (TD) and term-occurrence (TO) that correspond to the distances and occurrences of words in the context. By expressing the TDTO model in terms of a log-linear interpolation framework, the impact of the TD and TO towards the final estimation can be tuned. Specifically, as the TD events, i.e. positions, within a long context are likely rare or unseen, the weight of the TD component can be tuned down accordingly to alleviate the data scarcity problem.Through a series of experiments, the TDTO model has been shown to be capable of exploiting the long context to reduce the perplexities of the language models. On the BLLIP Wall Street Journal (WSJ) and Switchboard-1 (SWB) corpora, perplexity reductions up to 11.2% and 6.5% were obtained, with the context lengths of seven and eight, respectively. In addition, the TDTO model has been shown to outperform other conventional models used to exploit the long context, such as the distant-bigram, trigger and BOW models -the TDTO model consistently showed lower perplexities. The apxv plicability of the TDTO model has been examined on several tasks, such as the speech recognition, text classification and word prediction. The TDTO model has been shown to improve the baseline performance on all the considered tasks. Furthermore, this thesis proposes a neural network implementation of the TDTO model. The aim is to provide a better smoothing mechanism for TDTO modeling. The resulted model, referred to as the neural network based TDTO (NN-TDTO) model, has been empirically shown to outperform the baseline TDTO model in both perplexity and speech recognition accuracy. On the WSJ corpus, the NN-TDTO model yielded up to 9.2% lower perplexity as compared to the TDTO model. On the Aurora-4 speech recognition task, the NN-TDTO model obtained up to 12.9% relatively lower word error rate.

show abstract

RETRACTED ARTICLE: A novel semantic smoothing method based on log-bilinear model for bayesian text classification

Luo

Liu

2017

Multimed Tools Appl

View full text Add to dashboard Cite

A Term Association Translation Model for Naive Bayes Text Classification

Cited by 3 publications

References 18 publications

Decoupling Word-Pair Distance and Co-occurrence Information for Effective Long History Context Language Modeling

Decoupling Word-Pair Distance and Co-occurrence Information for Effective Long History Context Language Modeling

Exploiting long context using joint distance and occurrence information for language modeling

RETRACTED ARTICLE: A novel semantic smoothing method based on log-bilinear model for bayesian text classification

Contact Info

Product

Resources

About