A Targeted Estimation of Distribution Algorithm Compared to Traditional Methods in Feature Selection

Abstract

The Targeted Estimation of Distribution Algorithm (TEDA) introduces into an EDA/GA hybrid framework a ‘Targeting’ process, whereby the number of active genes, or ‘control points’, in a solution is driven in an optimal direction. For larger feature selection problems with over a thousand features, traditional methods such as forward and backward selection are inefficient. Traditional EAs may perform better but are slow to optimize if a problem is sufficiently noisy that most large solutions are equally ineffective and it is only when much smaller solutions are discovered that effective optimization may begin. By using targeting, TEDA is able to drive down the feature set size quickly and so speeds up this process. This approach was tested on feature selection problems with between 500 and 20,000 features using all of these approaches and it was confirmed that TEDA finds effective solutions significantly faster than the other approaches

Estimation of Distribution Algorithms in Machine Learning: A Survey

“…Pruning the structure of a multilayer perceptron [136] cGA, EcGA, BOA Design of a fully connected multilayer perceptron [137] UMDA Hyperparameters of a convolutional network [ Symbolic regression [157] UMDA Symbolic regression [158] Denoising autoencoder genetic programming Support vector regression [159] UMDA G c Feature subset selection Selective naive Bayes [161] EBNA Selective naive Bayes [162] EBNA Selective naive Bayes [163] cGA, EcGA, BOA Support vector machines [164] TEDA Logistic regression [165] UMDA G c…”

“…The naive Bayes classifier was also used in [162] for an empirical comparison among EBNA, two types of genetic algorithms and two greedy algorithms as sequential feature selection and sequential feature elimination; in [163], three EDAs, namely, cGA, EcGA, and BOA, were experimentally compared. In [164], a hybrid method consisting of a genetic algorithm and a UMDA, named TEDA, was applied to problems with tens of thousands of predictor variables. In [165], the EDA process (UMDA G c ) was embedded in an adapted recursive feature elimination procedure of a logistic regression model.…”