Genetic programming for feature construction and selection in classification on high-dimensional data

Tran, Binh; Xue, Bing; Zhang, Mengjie

doi:10.1007/s12293-015-0173-y

Cited by 153 publications

(57 citation statements)

References 33 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of GP for visualisation of solutions for production scheduling problems has also been recently investigated [13]. GP has also been applied to other tangential unsupervised learning tasks for feature creation, such as clustering [7], as well as extensive use in supervised learning domains [20,12]. Clearly, GP has shown significant potential as a feature construction method, and so it is hoped that it can be extended to directly perform manifold learning as well.…”

Section: Related Workmentioning

confidence: 99%

Can Genetic Programming Do Manifold Learning Too?

Lensen

Xue

Zhang

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Exploratory data analysis is a fundamental aspect of knowledge discovery that aims to find the main characteristics of a dataset. Dimensionality reduction, such as manifold learning, is often used to reduce the number of features in a dataset to a manageable level for human interpretation. Despite this, most manifold learning techniques do not explain anything about the original features nor the true characteristics of a dataset. In this paper, we propose a genetic programming approach to manifold learning called GP-MaL which evolves functional mappings from a high-dimensional space to a lower dimensional space through the use of interpretable trees. We show that GP-MaL is competitive with existing manifold learning algorithms, while producing models that can be interpreted and re-used on unseen data. A number of promising future directions of research are found in the process.

show abstract

Section: Related Workmentioning

confidence: 99%

Can Genetic Programming Do Manifold Learning Too?

Lensen

Xue

Zhang

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Feature construction for high-dimensional datasets is considered in [16], for eight bio-medical binary classification problems, with 2,000 to 24,188 features. This approach is different from the typical ones, as the authors propose to use SGP to evolve classifiers rather than features, and extract features from the components (subtrees) of such classifiers.…”

Section: Related Workmentioning

confidence: 99%

On explaining machine learning models by evolving crucial and compact features

Virgolin

Alderliesten

Bosman

2020

Swarm and Evolutionary Computation

View full text Add to dashboard Cite

Feature construction can substantially improve the accuracy of Machine Learning (ML) algorithms. Genetic Programming (GP) has been proven to be effective at this task by evolving non-linear combinations of input features. GP additionally has the potential to improve ML explainability since explicit expressions are evolved. Yet, in most GP works the complexity of evolved features is not explicitly bound or minimized though this is arguably key for explainability. In this article, we assess to what extent GP still performs favorably at feature construction when constructing features that are (1) Of small-enough number, to enable visualization of the behavior of the ML model; (2) Of small-enough size, to enable interpretability of the features themselves; (3) Of sufficient informative power, to retain or even improve the performance of the ML algorithm. We consider a simple feature construction scheme using three different GP algorithms, as well as random search, to evolve features for four ML algorithms, including support vector machines and random forest. Our results on 20 datasets pertaining to classification and regression problems show that constructing only two compact features can be sufficient to rival the use of the entire original feature set. We further find that a modern GP algorithm, GP-GOMEA, performs best overall. These results, combined with examples that we provide of readable constructed features and of 2D visualizations of ML behavior, lead us to positively conclude that GP-based feature construction still works well when explicitly searching for compact features, making it extremely helpful to explain ML models.

show abstract

“…FS and FC have seen wide application in supervised learning tasks [41,39], including EC-based FS [43] and FC [32,42]. FS and FC have also been used in unsupervised learning tasks [12], but little research has considered EC-based approaches for these tasks [2].…”

Section: Dimensionality Reductionmentioning

confidence: 99%

Multi-objective genetic programming for manifold learning: balancing quality and dimensionality

Lensen

Zhang

Xue

2020

Genet Program Evolvable Mach

Self Cite

View full text Add to dashboard Cite

Manifold learning techniques have become increasingly valuable as data continues to grow in size. By discovering a lower-dimensional representation (embedding) of the structure of a dataset, manifold learning algorithms can substantially reduce the dimensionality of a dataset while preserving as much information as possible. However, state-of-the-art manifold learning algorithms are opaque in how they perform this transformation. Understanding the way in which the embedding relates to the original high-dimensional space is critical in exploratory data analysis. We previously proposed a Genetic Programming method that performed manifold learning by evolving mappings that are transparent and interpretable. This method required the dimensionality of the embedding to be known a priori, which makes it hard to use when little is known about a dataset. In this paper, we substantially extend our previous work, by introducing a multi-objective approach that automatically balances the competing objectives of manifold quality and dimensionality. Our proposed approach is competitive with a range of baseline and state-of-the-art manifold learning methods, while also providing a range (front) of solutions that give different tradeoffs between quality and dimensionality. Furthermore, the learned models are shown to often be simple and efficient, utilising only a small number of features in an interpretable manner.

show abstract

Genetic programming for feature construction and selection in classification on high-dimensional data

Cited by 153 publications

References 33 publications

Can Genetic Programming Do Manifold Learning Too?

Can Genetic Programming Do Manifold Learning Too?

On explaining machine learning models by evolving crucial and compact features

Multi-objective genetic programming for manifold learning: balancing quality and dimensionality

Contact Info

Product

Resources

About