A Term Frequency Based Weighting Scheme Using Naïve Bayes for Text Classification

Abstract

Term weighting is a strategy to assign weights to terms to improve the performance of many classifiers, such as kNN and SVM in text classification. Supervised term weighting methods have received increasing attention, in which information on the membership of training documents to classes is used. Most existing methods follow the local weight multiplies the global weight framework, but the contribution of term frequency for term weighting has not been fully investigated. In this paper, we propose a weighting scheme named term frequency-relevance term frequency based on a probabilistic model. After investigating two kinds of widely used naïve Bayes (NB) models, we employ the term event Multinomial NB model to capture the term frequency information. The matching score function based on the prediction probability ratio can then be factorized. Finally, we get the weight for each term by replacing the parameter by an estimator, term frequency is used in formulating not only the local weight factor but also the global weight factor. Numerical experiment results on two benchmark text datasets (Reuters-21578 and 20 Newsgroups) demonstrate that our proposed method outperforms the representative term weighting methods.