Feature selection based on self-information and entropy measures for incomplete neighborhood decision systems

Abstract

For incomplete datasets with mixed numerical and symbolic features, feature selection based on neighborhood multi-granulation rough sets (NMRS) is developing rapidly. However, its evaluation function only considers the information contained in the lower approximation of the neighborhood decision, which easily leads to the loss of some information. To solve this problem, we construct a novel NMRS-based uncertain measure for feature selection, named neighborhood multi-granulation self-information-based pessimistic neighborhood multi-granulation tolerance joint entropy (PTSIJE), which can be used to incomplete neighborhood decision systems. First, from the algebra view, four kinds of neighborhood multi-granulation self-information measures of decision variables are proposed by using the upper and lower approximations of NMRS. We discuss the related properties, and find the fourth measure-lenient neighborhood multi-granulation self-information measure (NMSI) has better classification performance. Then, inspired by the algebra and information views simultaneously, a feature selection method based on PTSIJE is proposed. Finally, the Fisher score method is used to delete uncorrelated features to reduce the computational complexity for high-dimensional gene datasets, and a heuristic feature selection algorithm is raised to improve classification performance for mixed and incomplete datasets. Experimental results on 11 datasets show that our method selects fewer features and has higher classification accuracy than related methods.

Effective Attribute Reduction Algorithm Based on Fuzzy Uncertainties Using Shared Neighborhood Granulation

“…The used neighborhood re- lation is still crisp which may fail to reflect the actual certainty in data. 3) NSIFS [35]: It is a forward greedy approach that applies the neighborhood self-information and entropy measures. It extends some information measures to neighborhood-based ones and can describe the feature significance from an uncertain perspective.…”

Nested ensemble selection: An effective hybrid feature selection method

“…A filter method based on neighborhood multi-granulation rough sets is proposed in [52] that uses a novel self-information measure for initial preprocessing followed by Fisher score to delete uncorrelated features. Another filter method called the Highest Wins was proposed in [33] for intrusion detection.…”