A Novel Feature Selection Technique for Text Classification Using Naïve Bayes

Abstract

With the proliferation of unstructured data, text classification or text categorization has found many applications in topic classification, sentiment analysis, authorship identification, spam detection, and so on. There are many classification algorithms available. Naïve Bayes remains one of the oldest and most popular classifiers. On one hand, implementation of naïve Bayes is simple and, on the other hand, this also requires fewer amounts of training data. From the literature review, it is found that naïve Bayes performs poorly compared to other classifiers in text classification. As a result, this makes the naïve Bayes classifier unusable in spite of the simplicity and intuitiveness of the model. In this paper, we propose a two-step feature selection method based on firstly a univariate feature selection and then feature clustering, where we use the univariate feature selection method to reduce the search space and then apply clustering to select relatively independent feature sets. We demonstrate the effectiveness of our method by a thorough evaluation and comparison over 13 datasets. The performance improvement thus achieved makes naïve Bayes comparable or superior to other classifiers. The proposed algorithm is shown to outperform other traditional methods like greedy search based wrapper or CFS.

Automatic Irony Detection using Feature Fusion and Ensemble Classifier

“…Otherwise, MI value reaches the maximum when feature distribution is in intra-class only. The work presented in [26] suggested that features may convey similar information in the feature space. In conjecture to that, features evininge similar information are grouped to select the most representative features from each group.…”

“…The k-means clustering algorithm works iteratively to assign features to one of the k clusters based on the similar information features. To determine the optimal number of clusters (k) as mentioned in [26]…”

“…The conventional feature selection methods such as Chisquare (χ 2 ), Information Gain (IG), Mutual Information (MI) are used to select discriminative features from high dimensional feature spaces. However, the selected feature subset may have features which convey similar information [26]. On this line, we propose a two stage feature subset selection using conventional feature selection methods and a clustering method to select the most discriminative features from a high dimensional feature space.…”

Cluster Based Symbolic Representation for Skewed Text Categorization

“…A number of works can be traced in recent years addressing the problem of text classification through feature selection. Feature selection algorithms such as chisquare, information gain, and mutual information (Yang and Pedersen., 1997) though seem to be powerful techniques for text data, a number of novel feature selection algorithms based on genetic algorithm (Bharti and Singh., 2016;Ghareb et al, 2016), ant colony optimization (Dadaneh et al, 2016;Moradi and Gholampour., 2016;Uysal., 2016;Meena et al, 2012), Bayesian principle Zhang et al, 2016;Feng et al, 2012;Fenga et al, 2015;Sarkar et al, 2014), clustering of features (Bharti and Singh., 2015), global information gain (Shang et al, 2013), adaptive keyword (Tasci and Gungor., 2013), global ranking (Pinheiro et al, 2012;Pinheiro et al, 2015) are proposed.…”

A Neural Network Language Document Representation Technique for Web-Page Classification

“…To achieve high classification result of the Web Page Classification (WPC) system, an excellent representation of textual data (Preprocessing/DR) should contain as much information as possible from the original document [8]. Also, the accuracy of most classification algorithms depends on the quality and size of training data which is inherently dependent on the document representation technique [9]. Several researchers have contributed to the document representation stage of the web page classification system because irrelevant and redundant features often degrade the performance of the classification algorithms both in speed and classification accuracy and also its tendency to reduce overfitting [10].…”