A Fast Non-Redundant Feature Selection Technique for Text Data

Abstract

Feature selection is critical in reducing the size of data and improving classifier accuracy by selecting an optimum subset of the overall features. Traditionally, each feature is given a score against a particular category (such as using Mutual Information) and the task of feature selection comes down to choosing the top k ranked features with the best average score across all categories. However, this approach has two major drawbacks. Firstly, the maximum or average score of a feature with a class might not necessarily determine its discriminating strength among samples of other classes. Secondly, most feature selection methods only use the scores to select the discriminating features from the corpus without taking into account the redundancy of information provided by the selected features. In this paper, we propose a new feature ranking score measure called the Discriminative Mutual Information (DMI) score. This score helps to select features that distinguish samples of one category against all other categories. Moreover, Non-Redundant Feature Selection (NRFS) heuristic is also proposed that explicitly takes the problem of feature redundancy into account when selecting the features set. The performance of our approach is investigated and compared with other feature selection techniques on datasets derived from high-dimensional text corpora using multiple classification algorithms. The results show that the proposed method leads to better classification micro-F1 score as compared to other state-of-the-art methods. In particular, the proposed method shows great improvement when the number of selected features are small as well as an overall higher robustness to label noise.

Improved meta‐heuristic algorithm for selecting optimal features: A big data classification model

“…The within-class scatter matrix Wid class and between-class scatter matrix Bet class is being used to overcome the problems associated with the ideal discrimination projection matrix. Furthermore, Equation (11) depicts the mathematical equation used to calculate the projection matrix. In addition, the formulas for evaluating Bet class and Wid class are represented by the mathematical equations Equation (12) and Equation (13), respectively.…”

“…In addition, the formulas for evaluating Bet class and Wid class are represented by the mathematical equations Equation (12) and Equation (13), respectively. The eigenvectors of s are shown in Equation (11). The projection matrix is denoted by the symbol S and the eigenvectors of S are shown in Equation ( 14), in which G T = Bet class + Wid class ,, F J is the feature vector of the data, 𝛼 N and N is the data vector and samples in the data class J.…”

“…Recently, with the adverse growth of big data in all science and engineering domains especially in physical, biological, and biomedical sciences, there is faster development of networking, data collection, and data storage capacity. [1][2][3][4][5][6][7][8][9][10][11] Data mining refers to be the activity of searching relevant or pertinent information by processing the big datasets. This has greatly influenced in the decision-making process.…”

“…As a result, there should be some system in place to help organize unstructured data allowing the user to quickly access the information they require. Recently, with the adverse growth of big data in all science and engineering domains especially in physical, biological, and biomedical sciences, there is faster development of networking, data collection, and data storage capacity 1–11 . Data mining refers to be the activity of searching relevant or pertinent information by processing the big datasets.…”

Weighted multi-view co-clustering (WMVCC) for sparse data

“…This is particularly true for sparse data. Feature selection is a commonly used approach to select only relevant (discriminatory) features [8,9] which help in increasing the effectiveness of an algorithm as well as reducing its complexity. This is essential since all features are not equally important and irrelevant features negatively impact the clusters.…”