Genetic programming for multiple-feature construction on high-dimensional classification

Tran, Binh; Xue, Bing; Zhang, Mengjie

doi:10.1016/j.patcog.2019.05.006

Cited by 86 publications

(43 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The authors of [43] study whether modern model-based GP can be useful when particularly compact symbolic regression solutions are sought, to allow interpretability. A very different take to enable or improve interpretability is taken in [22,41,45], where interpretability is sought by means of feature construction and dimensionality reduction. In [22] in particular, MOGP is used, with solution size as a simple PHI.…”

Section: Related Workmentioning

confidence: 99%

Learning a Formula of Interpretability to Learn Interpretable Formulas

Virgolin

Lorenzo

Medvet

et al. 2020

Parallel Problem Solving From Nature – PPSN XVI

View full text Add to dashboard Cite

Many risk-sensitive applications require Machine Learning (ML) models to be interpretable. Attempts to obtain interpretable models typically rely on tuning, by trial-and-error, hyper-parameters of model complexity that are only loosely related to interpretability. We show that it is instead possible to take a meta-learning approach: an ML model of non-trivial Proxies of Human Interpretability (PHIs) can be learned from human feedback, then this model can be incorporated within an ML training process to directly optimize for interpretability. We show this for evolutionary symbolic regression. We first design and distribute a survey finalized at finding a link between features of mathematical formulas and two established PHIs, simulatability and decomposability. Next, we use the resulting dataset to learn an ML model of interpretability. Lastly, we query this model to estimate the interpretability of evolving solutions within bi-objective genetic programming. We perform experiments on five synthetic and eight real-world symbolic regression problems, comparing to the traditional use of solution size minimization. The results show that the use of our model leads to formulas that are, for a same level of accuracy-interpretability trade-off, either significantly more or equally accurate. Moreover, the formulas are also arguably more interpretable. Given the very positive results, we believe that our approach represents an important stepping stone for the design of next-generation interpretable (evolutionary) ML algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning a Formula of Interpretability to Learn Interpretable Formulas

Virgolin

Lorenzo

Medvet

et al. 2020

Parallel Problem Solving From Nature – PPSN XVI

View full text Add to dashboard Cite

show abstract

“…FCS is iterative, and this can lead to suboptimal performance for a chosen K, compared to attempting to find K features at once. This is because the contributions of multiple features to an ML algorithm are not necessarily perpendicular to each other [23]. FCS could be changed to find at any given iteration, a synergistic set of K features, that is independent from previous iterations.…”

Section: Discussionmentioning

confidence: 99%

On explaining machine learning models by evolving crucial and compact features

Virgolin

Alderliesten

Bosman

2020

Swarm and Evolutionary Computation

View full text Add to dashboard Cite

Feature construction can substantially improve the accuracy of Machine Learning (ML) algorithms. Genetic Programming (GP) has been proven to be effective at this task by evolving non-linear combinations of input features. GP additionally has the potential to improve ML explainability since explicit expressions are evolved. Yet, in most GP works the complexity of evolved features is not explicitly bound or minimized though this is arguably key for explainability. In this article, we assess to what extent GP still performs favorably at feature construction when constructing features that are (1) Of small-enough number, to enable visualization of the behavior of the ML model; (2) Of small-enough size, to enable interpretability of the features themselves; (3) Of sufficient informative power, to retain or even improve the performance of the ML algorithm. We consider a simple feature construction scheme using three different GP algorithms, as well as random search, to evolve features for four ML algorithms, including support vector machines and random forest. Our results on 20 datasets pertaining to classification and regression problems show that constructing only two compact features can be sufficient to rival the use of the entire original feature set. We further find that a modern GP algorithm, GP-GOMEA, performs best overall. These results, combined with examples that we provide of readable constructed features and of 2D visualizations of ML behavior, lead us to positively conclude that GP-based feature construction still works well when explicitly searching for compact features, making it extremely helpful to explain ML models.

show abstract

“…In [44], [45], [53], one high-level feature was constructed by GP using the original features and the experimental results showed that the constructed feature can improve the classification accuracy. In [54], [55], [56], GP was employed to construct multiple features and the results showed that the constructed features achieve better classification performance than the methods using the original features and the single constructed feature. However, when constructing multiple features, it is necessary to set the number of constructed features, which requires domain knowledge.…”

Section: B Gp For Feature Extraction and Constructionmentioning

confidence: 99%

Automatic Feature Extraction and Construction Using Genetic Programming for Rotating Machinery Fault Diagnosis

Peng

Wan

et al. 2021

IEEE Trans. Cybern.

Self Cite

View full text Add to dashboard Cite

Feature extraction is an essential process in the intelligent fault diagnosis of rotating machinery. Although existing feature extraction methods can obtain representative features from the original signal, domain knowledge and expert experience are often required. In this paper, a novel diagnosis approach based on evolutionary learning, namely automatic feature extraction and construction using genetic programming (AFECGP), is proposed to automatically generate informative and discriminative features from original vibration signals for identifying different fault types of rotating machinery. To achieve this, a new program structure, a new function set and a new terminal set are developed in AFECGP to allow it to detect important subband signals, extract and construct informative features, automatically and simultaneously. More importantly, AFECGP can produce a flexible number of features for classification. Having the generated features, k-Nearest Neighbors is employed to perform fault diagnosis. The performance of the AFECGP based fault diagnosis approach is evaluated on four fault diagnosis datasets of varying difficulty and compared with fourteen baseline methods. The results show that the proposed approach achieves better fault diagnosis accuracy on all the datasets than the competitive methods and can effectively identify different fault conditions of rolling bearing, gear, and rotor.

show abstract

Genetic programming for multiple-feature construction on high-dimensional classification

Cited by 86 publications

References 40 publications

Learning a Formula of Interpretability to Learn Interpretable Formulas

Learning a Formula of Interpretability to Learn Interpretable Formulas

On explaining machine learning models by evolving crucial and compact features

Automatic Feature Extraction and Construction Using Genetic Programming for Rotating Machinery Fault Diagnosis

Contact Info

Product

Resources

About