A Novel Nonparametric Feature Selection Approach Based on Mutual Information Transfer Network

Abstract

The filter feature selection algorithm is habitually used as an effective way to reduce the computational cost of data analysis by selecting and implementing only a subset of original features into the study. Mutual information (MI) is a popular measurement adopted to quantify the dependence among features. MI-based greedy forward methods (MIGFMs) have been widely applied to escape from computational complexity and exhaustion of high-dimensional data. However, most MIGFMs are parametric methods that necessitate proper preset parameters and stopping criteria. Improper parameters may lead to ignorance of better results. This paper proposes a novel nonparametric feature selection method based on mutual information and mixed-integer linear programming (MILP). By forming a mutual information network, we transform the feature selection problem into a maximum flow problem, which can be solved with the Gurobi solver in a reasonable time. The proposed method attempts to prevent negligence on obtaining a superior feature subset while keeping the computational cost in an affordable range. Analytical comparison of the proposed method with six feature selection methods reveals significantly better results compared to MIGFMs, considering classification accuracy.

Benign and Malignant Breast Tumor Classification in Ultrasound and Mammography Images via Fusion of Deep Learning and Handcraft Features

“…Mutual information is based on Shannon entropy, which measures the dependence or mutual information between two random variables (X, Y) [50][51][52]. It measures the amount of information obtained about one random variable by observing another variable; in other words, it determines how much we can know about one variable by taking into account another.…”

Time Series Feature Selection Method Based on Mutual Information

“…This is an unsupervised feature selection method and has become an important feature selection method [29]. In combination with other methods, MI-based feature selection has derived many other methods, such as MI with a correlation coefficient [30], variance impact factor [31], fisher score [32], binary butterfly optimization algorithm [33], conditional mutual information [34], deep neural network [35,36], etc. Among them, the authors of [37] proposed a deep generative network model for feature extraction of multivariate time series and introduced mutual information into the loss function to improve the expression capability and accuracy of the model.…”

Short-Term Electrical Load Forecasting Based on Fuzzy Rough Set Feature Selection and Multi-kernel Extreme Learning Machine

“…Filter methods focus on statistical characteristics of features, using an evaluation of the correlation between features and the prediction target for selection. Examples include correlation analysis (CA) [23], fuzzy rough sets (FRS) [24], and the mutual information (MI) method [25,26]. The CA method identifies features that are highly correlated with target variables by measuring the linear correlation between each feature and the target variable.…”

“…CA is then employed to assess the correlation among the initially screened features, thereby eliminating redundant features. Compared with the traditional filter methods, the FRSCA method proposed in this paper better captures non-linear relationships and interactions among features, reduces computational complexity and the risk of overfitting compared to wrapper methods, and improves interpretability and universality of feature selection compared with embedded methods [26,29].…”