Identifying the most informative features using a structurally interacting elastic net

Abstract

Feature selection can efficiently identify the most informative features with respect to the target feature used in training. However, state-of-the-art vector-based methods are unable to encapsulate the relationships between feature samples into the feature selection process, thus leading to significant information loss. To address this problem, we propose a new graph-based structurally interacting elastic net method for feature selection. Specifically, we commence by constructing feature graphs that can incorporate pairwise relationship between samples. With the feature graphs to hand, we propose a new information theoretic criterion to measure the joint relevance of different pairwise feature combinations with respect to the target feature graph representation. This measure is used to obtain a structural interaction matrix where the elements represent the proposed information theoretic measure between feature pairs. We then formulate a new optimization model through the combination of the structural interaction matrix and an elastic net regression model for the feature subset selection problem. This allows us to a) preserve the information of the original vectorial space, b) remedy the information loss of the original feature space caused by using graph representation, and c) promote a sparse solution and also encourage correlated features to be selected. Because the proposed optimization problem is non-convex, we develop an efficient alternating direction multiplier method (ADMM) to locate the optimal solutions. Extensive experiments on various datasets demonstrate the effectiveness of the proposed method.

A Hybrid Feature Selection Optimization Model for High Dimension Data Classification

“…It is also shown that AEN-CMI contributes to the adaptive clustering effect by assessing the significance of gene ranking. Lixin Cui and Yue Wang [15] suggested a technique that can encapsulate the structural correlation between feature methods in the feature selection process, and display features in the form of graphics and vertices. Therefore, the obtained information matrix is utilised to produce an optimization model to determine the object with the greatest relevance and least redundancy to the objective function.…”

A Novel Ensemble Credit Scoring Model Based on Extreme Learning Machine and Generalized Fuzzy Soft Sets

“…Although Lasso has been proved to be easily interpretable and effective under various circumstances, it still has some shortcomings [18]. Zou and Hastie [19] put forward an expansion approach called elastic net to conduct selection.…”

Single image super resolution via neighbor reconstruction

“…Neighbor Embedding (NE) approaches assume that features which are drawn from small low-and high-resolution patches lie on two local geometrically similar manifolds (Wang et al, 2019;Bai et al, 2014Bai et al, , 2018. Based on this assumption NE approaches reconstruct high-resolution features with local geometric structure recording coefficients which are shared in lowresolution space (Liu and Bai, 2012;Cui et al, 2017Cui et al, , 2019. A representative NE approach is A+ method proposed by Timofte et al (Timofte et al, 2014).…”