Feature Selection for Classificatory Analysis Based on Information-theoretic Criteria

Huang, Jinjie; Lv, Ning; Li, Shuangquan; Cai, Yunze

doi:10.3724/sp.j.1004.2008.00383

Cited by 17 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the interaction between the three relevant attributes for the class variable A 1 ; A 2 and A 4 is not captured and the attributes A 1 and A 4 are wrongly deleted. Liu et al [34] extends the work of Huang et al [51] by expanding the concept of mutual information between a feature and the class variable given the rest of the features in the classifier. However, their proposed heuristics, at first, select attributes relevant only for the class variable, and therefore, again, for our example, the attributes A 1 and A 4 that are irredundant for the classification task but redundant for C at level 0 are eliminated.…”

Section: Related Workmentioning

confidence: 97%

“…In fact (conditional) mutual information and its variants are rather popular methods for feature selection used in many recent papers. Huang et al [51], for example, introduce some parameters to learn from data when attributes are relevant or irredundant for the class variable. Recall that we use the MDL's penalty term to deal with noisy datasets.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Feature selection for Bayesian network classifiers using the MDL-FS score

Drugan

Wiering

2010

International Journal of Approximate Reasoning

View full text Add to dashboard Cite

When constructing a Bayesian network classifier from data, the more or less redundant features included in a dataset may bias the classifier and as a consequence may result in a relatively poor classification accuracy. In this paper, we study the problem of selecting appropriate subsets of features for such classifiers. To this end, we propose a new definition of the concept of redundancy in noisy data. For comparing alternative classifiers, we use the Minimum Description Length for Feature Selection (MDL-FS) function that we introduced before. Our function differs from the well-known MDL function in that it captures a classifier's conditional log-likelihood. We show that the MDL-FS function serves to identify redundancy at different levels and is able to eliminate redundant features from different types of classifier. We support our theoretical findings by comparing the featureselection behaviours of the various functions in a practical setting. Our results indicate that the MDL-FS function is more suited to the task of feature selection than MDL as it often yields classifiers of equal or better performance with significantly fewer attributes.

show abstract

Section: Related Workmentioning

confidence: 97%

Section: Related Workmentioning

confidence: 99%

Feature selection for Bayesian network classifiers using the MDL-FS score

Drugan

Wiering

2010

International Journal of Approximate Reasoning

View full text Add to dashboard Cite

show abstract

“…A number of efforts have been made to build a well-organized and successful feature selection algorithms based on the MI idea [1], [3], [9], [11]. In this paper proposes a feature selection algorithm based on the MI concept to suggest enhancement over these past efforts [5], [6]. The improvement is based on the redundancy criteria.…”

Section: Introductionmentioning

confidence: 99%

An improved feature selection algorithm with conditional mutual information for classification problems

Jaganathan

Ramasamy

2013

2013 International Conference on Human Computer Interactions (ICHCI)

View full text Add to dashboard Cite

The purpose of the feature selection is to eliminate insignificant features from entire dataset and simultaneously to keep the class discriminatory information for classification problems. Many feature selection algorithms have been proposed to measure the relevance and redundancy of the features and class variables. In this paper, we proposed an improved feature selection algorithm based on maximum relevance and minimum redundancy criterion. The relevance of a feature to the class variables are evaluated with mutual information and conditional mutual information is used to calculate the redundancy between the selected and the candidate features to each class variable. The experimental result is tested with five benchmarked datasets available from UCI Machine Learning Repository. The results shows the proposed algorithm is considered quite well when compared with some existing algorithms.

show abstract

“…Filter method is a feature selection technique that adopting a specific evaluation criterion to select features, which is independent of inductive learning algorithms. The conventional evaluation criterions of Filter method adopted are: χ 2 -test [2] , information entropy [14] , mutual information [4] , minimum joint mutual information loss [15] , minimum classification error [6] , etc.…”

Section: Introductionmentioning

confidence: 99%

An Improved Feature Selection Algorithm Based on Parzen Window and Conditional Mutual Information

Wen

Chen

et al. 2013

AMM

View full text Add to dashboard Cite

In this paper, an improved feature selection algorithm by conditional mutual information with Parzen window was proposed, which adopted conditional mutual information as an evaluation criterion of feature selection in order to overcome the deficiency of feature redundant and used Parzen window to estimate the probability density functions and calculate the conditional mutual information of continuous variables, in such a way as to achieve feature selection for continuous data.

show abstract

Feature Selection for Classificatory Analysis Based on Information-theoretic Criteria

Cited by 17 publications

References 29 publications

Feature selection for Bayesian network classifiers using the MDL-FS score

Feature selection for Bayesian network classifiers using the MDL-FS score

An improved feature selection algorithm with conditional mutual information for classification problems

An Improved Feature Selection Algorithm Based on Parzen Window and Conditional Mutual Information

Contact Info

Product

Resources

About