Adaptive Feature Selection Based on the Most Informative Graph-Based Features

Abstract

Abstract. In this paper, we propose a novel method to adaptively select the most informative and least redundant feature subset, which has strong discriminating power with respect to the target label. Unlike most traditional methods using vectorial features, our proposed approach is based on graph-based features and thus incorporates the relationships between feature samples into the feature selection process. To efficiently encapsulate the main characteristics of the graphbased features, we probe each graph structure using the steady state random walk and compute a probability distribution of the walk visiting the vertices. Furthermore, we propose a new information theoretic criterion to measure the joint relevance of different pairwise feature combinations with respect to the target feature, through the Jensen-Shannon divergence measure between the probability distributions from the random walk on different graphs. By solving a quadratic programming problem, we use the new measure to automatically locate the subset of the most informative features, that have both low redundancy and strong discriminating power. Unlike most existing state-of-the-art feature selection methods, the proposed information theoretic feature selection method can accommodate both continuous and discrete target features. Experiments on the problem of P2P lending platforms in China demonstrate the effectiveness of the proposed method.

Identifying the most informative features using a structurally interacting elastic net

“…This method can avoid discarding some valuable features arising in individual feature combinations. g) GF-RW [27]: Is a graph-based feature selection method which incorporates pairwise relationship between samples of each feature dimension.…”

“…The most relevant vectorial features are located by selecting the graph-based features that are most similar to the graph-based target feature, in terms of the Jensen-Shannon divergence measure between the graphs. To adaptively determine the most relevant feature subset, Cui et al [27] have further developed a new information theoretic feature selection method which a) encapsulates the relationship between sample pairs for each feature dimension and b) automatically identifies the subset containing the most informative and least redundant features by solving a quadratic programming problem.…”

“…To illustrate this point, assume that two pairs of samples from the same feature dimension f i are denoted as {f i1 , f i3 } and {f i3 , f i5 }, respectively. Following Cui et al [27], we transform the feature vector f i into a graph-based representation G i , which is a complete weighted graph. Each vertex v a of G i represents a corresponding sample f ia in f i and each weighted edge (v a , v b ) represents the relationship between sample pair f ia and f ib .…”

Single image super resolution via neighbor reconstruction

“…Neighbor Embedding (NE) approaches assume that features which are drawn from small low-and high-resolution patches lie on two local geometrically similar manifolds (Wang et al, 2019;Bai et al, 2014Bai et al, , 2018. Based on this assumption NE approaches reconstruct high-resolution features with local geometric structure recording coefficients which are shared in lowresolution space (Liu and Bai, 2012;Cui et al, 2017Cui et al, , 2019. A representative NE approach is A+ method proposed by Timofte et al (Timofte et al, 2014).…”

Mutual information based feature subset selection in multivariate time series classification

“…Most of these methods are based on transforming the original TS using a di↵erent representation. For instance, methods such as shapelets [22], symbolic dynamic methods [23], pseudo-observations [24] are implemented, while others extract features as graph-based features [25], pairwise mutual information [26] (no details are given for the computation of the MI between TS) or correlations [27]. Recently, in [28], for the extraction and the selection of relevant and non-redundant multivariate ordinal patterns for classification, a technique called Ordex is presented.…”