Recent advances and emerging challenges of feature selection in the context of big data

Bolón-Canedo, Verónica; Sánchez-Maroño, Noelia; Alonso‐Betanzos, Amparo

doi:10.1016/j.knosys.2015.05.014

Cited by 253 publications

(123 citation statements)

References 115 publications

Supporting

Mentioning

113

Contrasting

Unclassified

Order By: Relevance

“…In fact, MI(X i , S) = MI(X disc , X) = 0.693, but JMI approximates it by a smaller value, i.e. 1 2 MI(X disc , X) + 1 2 MI(X disc , X − k ′ Y ) = 1 2 × 0.693 + 1 2 × 0.686 = 0.6895. Similarly, MI(X disc , S|C k ) = MI(X disc , X|C k ) = 0.693, but again JMI approximates it by a smaller value, i.e.…”

Section: Methodsmentioning

confidence: 99%

Theoretical foundations of forward feature selection methods based on mutual information

et al. 2019

View full text Add to dashboard Cite

Feature selection problems arise in a variety of applications, such as microarray analysis, clinical prediction, text categorization, image classification and face recognition, multi-label learning, and classification of internet traffic. Among the various classes of methods, forward feature selection methods based on mutual information have become very popular and are widely used in practice. However, comparative evaluations of these methods have been limited by being based on specific datasets and classifiers. In this paper, we develop a theoretical framework that allows evaluating the methods based on their theoretical properties. Our framework is grounded on the properties of the target objective function that the methods try to approximate, and on a novel categorization of features, according to their contribution to the explanation of the class; we derive upper and lower bounds for the target objective function and relate these bounds with the feature types. Then, we characterize the types of approximations taken by the methods, and analyze how these approximations cope with the good properties of the target objective function. Additionally, we develop a distributional setting designed to illustrate the various deficiencies of the methods, and provide several examples of wrong feature selections. Based on our work, we identify clearly the methods that should be avoided, and the methods that currently have the best performance.Recently, there has been several attempts to undergo a theoretical evaluation of forward feature selection methods based on MI.[17] and [18] provide an interpretation of the objective function of actual methods as approximations of a target objective function, which is similar to ours. However, they do not study the consequences of these approximations from a theoretical point-of-view, i.e. how the various types of approximations affect the good properties of the target objective function, which is the main contribution of our work. Moreover, they do not cover all types of feature selection methods currently proposed. [26] evaluated methods based on a distributional setting similar to ours, but the analysis is restricted to the group of methods that ignore complementarity, and again, does not address the theoretical properties of the methods.The rest of the paper is organized as follows. We introduce some background on entropy and MI in Section 2. This is followed, in Section 3, by the presentation of the main concepts associated with conditional MI and MI between three random vectors. In Section 4, we focus on explaining the general context concerning forward feature selection methods based on MI, namely the target objective function, the categorization of features, and the relation between the feature types and the bounds of the target objective function. In Section 5, we introduce representative feature selection methods based on MI, along with their properties and drawbacks. In Section 6, we present a distribution based setting where some of the main drawbacks of the representative...

show abstract

Section: Methodsmentioning

confidence: 99%

Theoretical foundations of forward feature selection methods based on mutual information

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Feature selection has been widely studied in machine learning and pattern analysis. The topic of feature selection has been reviewed in a number of recent papers [36], [37] and [38]. In this section, we briefly state-of-the-art MI-based and sparse feature selection methods, which are related to our proposed method.…”

Section: Related Workmentioning

confidence: 99%

Identifying the most informative features using a structurally interacting elastic net

et al. 2019

View full text Add to dashboard Cite

Feature selection can efficiently identify the most informative features with respect to the target feature used in training. However, state-of-the-art vector-based methods are unable to encapsulate the relationships between feature samples into the feature selection process, thus leading to significant information loss. To address this problem, we propose a new graph-based structurally interacting elastic net method for feature selection. Specifically, we commence by constructing feature graphs that can incorporate pairwise relationship between samples. With the feature graphs to hand, we propose a new information theoretic criterion to measure the joint relevance of different pairwise feature combinations with respect to the target feature graph representation. This measure is used to obtain a structural interaction matrix where the elements represent the proposed information theoretic measure between feature pairs. We then formulate a new optimization model through the combination of the structural interaction matrix and an elastic net regression model for the feature subset selection problem. This allows us to a) preserve the information of the original vectorial space, b) remedy the information loss of the original feature space caused by using graph representation, and c) promote a sparse solution and also encourage correlated features to be selected. Because the proposed optimization problem is non-convex, we develop an efficient alternating direction multiplier method (ADMM) to locate the optimal solutions. Extensive experiments on various datasets demonstrate the effectiveness of the proposed method.

show abstract

“…the features (horizontal dimension) of data sets. This "big dimensionality" [2] is usually called the first V ("volume") of the 5 big data attributes. The two most common attributes, in addition to volume, are "velocity" and "variety" (see e.g.…”

Section: Big Data and Industrial Analyticsmentioning

confidence: 99%

Fuzzy Entropy Used for Predictive Analytics

Carlsson

Heikkilä

Mezei

2016

Fuzzy Logic in Its 50th Year

View full text Add to dashboard Cite

Process interruptions in (very) large production systems are difficult to deal with. Modern processes are highly automated; data is collected with sensor technology that forms a big data context and offers challenges to identify coming failures from the very large sets of data. The sensors collect huge amounts of data but the failure events are few and infrequent and hard to find (and even harder to predict). In this article, our goal is to develop models for predictive maintenance in a big data environment. The purpose of feature selection in the context of predictive maintenance is to identify a small set of process diagnostics that are sufficient to predict future failures. We apply interval-valued fuzzy sets and various entropy measures defined on them to perform feature selection on process diagnostics. We show how these models can be utilized as the basis of decision support systems in process industries to aid predictive maintenance.

show abstract

Recent advances and emerging challenges of feature selection in the context of big data

Cited by 253 publications

References 115 publications

Theoretical foundations of forward feature selection methods based on mutual information

Theoretical foundations of forward feature selection methods based on mutual information

Identifying the most informative features using a structurally interacting elastic net

Fuzzy Entropy Used for Predictive Analytics

Contact Info

Product

Resources

About