A parallel and distributed semantic Genetic Programming system

Vanneschi, Leonardo; Galvao, Bernardo

doi:10.1109/cec.2017.7969304

“…For this reason, this study takes into account different approaches that we expect to be beneficial in increasing the generalization ability of the ensemble model. The first idea comes from recent literature [34,37] where authors demonstrated that a blend of individuals created with standard syntax-based GP (STGP) and Geometric Semantic Genetic Programming (GSGP) results in a model with a better generalization ability with respect to the use of only one kind of solutions. With this in mind, to build the ensemble of GP models we run in parallel different GP populations, where some of them are evolved using STGP and others are evolved using GSGP (as done in [37]).…”

Section: Introductionmentioning

confidence: 99%

“…The first idea comes from recent literature [34,37] where authors demonstrated that a blend of individuals created with standard syntax-based GP (STGP) and Geometric Semantic Genetic Programming (GSGP) results in a model with a better generalization ability with respect to the use of only one kind of solutions. With this in mind, to build the ensemble of GP models we run in parallel different GP populations, where some of them are evolved using STGP and others are evolved using GSGP (as done in [37]). One of the hypotheses of this study is that a blend of STGP and GSGP is beneficial (in terms of generalization) also in building an ensemble of GP models.…”

Section: Introductionmentioning

confidence: 99%

Pruning Techniques for Mixed Ensembles of Genetic Programming Models

Castelli

¹

,

Gonçalves

²

,

Manzoni

³

et al. 2018

Lecture Notes in Computer Science

Self Cite

5

0

View full text Add to dashboard Cite

The objective of this paper is to define an effective strategy for building an ensemble of Genetic Programming (GP) models. Ensemble methods are widely used in machine learning due to their features: they average out biases, they reduce the variance and they usually generalize better than single models. Despite these advantages, building ensemble of GP models is not a well-developed topic in the evolutionary computation community. To fill this gap, we propose a strategy that blends individuals produced by standard syntax-based GP and individuals produced by geometric semantic genetic programming, one of the newest semantics-based method developed in GP. In fact, recent literature showed that combining syntax and semantics could improve the generalization ability of a GP model. Additionally, to improve the diversity of the GP models used to build up the ensemble, we propose different pruning criteria that are based on correlation and entropy, a commonly used measure in information theory. Experimental results, obtained over different complex problems, suggest that the pruning criteria based on correlation and entropy could be effective in improving the generalization ability of the ensemble model and in reducing the computational burden required to build it.

show abstract

“…In [239], a parallel and distributed GP system called Multi-Population Hybrid GP (MPHGP) is presented. In this system, two sub populations run in parallel: one for a Multi Objectives GP (MO-GP) algorithm and the other sub population for Geometric Semantic GP (GSGP).…”

Section: Ec-based Symbolic Regressionmentioning

confidence: 99%

Genetic Programming for Symbolic Regression on Incomplete Data

Al-Helali¹

0

View full text Add to dashboard Cite

Symbolic regression is the process of constructing mathematical expressions that best fit given data sets, where a target variable is expressed in terms of input variables. Unlike traditional regression methods, which optimise the parameters of pre-defined models, symbolic regression learns both the model structure and its parameters simultaneously. Genetic programming (GP) is a biologically-inspired evolutionary algorithm, that automatically generates computer programs to solve a given task. The flexible representation of GP along with its ``white box" nature makes it a dominant method for symbolic regression. Moreover, GP has been successfully employed for different learning tasks such as feature selection and transfer learning. Data incompleteness is a pervasive problem in symbolic regression, and machine learning in general, especially when dealing with real-world data sets. One common approach to handling data missingness is data imputation. Data imputation is the process of estimating missing values based on existing data. Another approach to deal with incomplete data is to build learning algorithms that directly work with missing values. Although a number of methods have been proposed to tackle the data missingness issue in machine learning, most studies focus on classification tasks. Little attention has been paid to symbolic regression on incomplete data. The existing symbolic regression methods are only applicable when the given data set is complete. The overall goal of the thesis is to improve the performance of symbolic regression on incomplete data by using GP for data imputation, instance selection, feature selection, and transfer learning. This thesis develops an imputation method to handle missing values for symbolic regression. The method integrates the instance-based similarity of the k-nearest neighbour method with the feature-based predictability of GP to estimate the missing values. The results show that the proposed method outperforms existing popular imputation methods. This thesis develops an instance selection method for improving imputation for symbolic regression on incomplete data. The proposed method has the ability to simultaneously build imputation and symbolic regression models such that the performance is improved. The results show that involving instance selection with imputation advances the performance of using the imputation alone. High-dimensionality is a serious data challenge, which is even more difficult on incomplete data. To address this problem in symbolic regression tasks, this thesis develops a feature selection method that can select a good set of features directly from incomplete data. The method not only improves the regression accuracy, but also enhances the efficiency of symbolic regression on high-dimensional incomplete data. Another challenging problem is data shortage. This issue is even more challenging when the data is incomplete. To handle this situation, this thesis develops transfer learning methods to improve symbolic regression in domains with incomplete and limited data. These methods utilise two powerful abilities of GP: feature construction and feature selection. The results show the ability of these methods to achieve positive transfer learning from domains with complete data to different (but related) domains with incomplete data. In summary, the thesis develops a range of approaches to improving the effectiveness and efficiency of symbolic regression on incomplete data by developing a number of GP-based methods. The methods are evaluated using different types of data sets considering various missingness and learning scenarios.

show abstract

Genetic Programming for Symbolic Regression on Incomplete Data

Al-Helali¹

0

View full text Add to dashboard Cite

Symbolic regression is the process of constructing mathematical expressions that best fit given data sets, where a target variable is expressed in terms of input variables. Unlike traditional regression methods, which optimise the parameters of pre-defined models, symbolic regression learns both the model structure and its parameters simultaneously. Genetic programming (GP) is a biologically-inspired evolutionary algorithm, that automatically generates computer programs to solve a given task. The flexible representation of GP along with its ``white box" nature makes it a dominant method for symbolic regression. Moreover, GP has been successfully employed for different learning tasks such as feature selection and transfer learning. Data incompleteness is a pervasive problem in symbolic regression, and machine learning in general, especially when dealing with real-world data sets. One common approach to handling data missingness is data imputation. Data imputation is the process of estimating missing values based on existing data. Another approach to deal with incomplete data is to build learning algorithms that directly work with missing values. Although a number of methods have been proposed to tackle the data missingness issue in machine learning, most studies focus on classification tasks. Little attention has been paid to symbolic regression on incomplete data. The existing symbolic regression methods are only applicable when the given data set is complete. The overall goal of the thesis is to improve the performance of symbolic regression on incomplete data by using GP for data imputation, instance selection, feature selection, and transfer learning. This thesis develops an imputation method to handle missing values for symbolic regression. The method integrates the instance-based similarity of the k-nearest neighbour method with the feature-based predictability of GP to estimate the missing values. The results show that the proposed method outperforms existing popular imputation methods. This thesis develops an instance selection method for improving imputation for symbolic regression on incomplete data. The proposed method has the ability to simultaneously build imputation and symbolic regression models such that the performance is improved. The results show that involving instance selection with imputation advances the performance of using the imputation alone. High-dimensionality is a serious data challenge, which is even more difficult on incomplete data. To address this problem in symbolic regression tasks, this thesis develops a feature selection method that can select a good set of features directly from incomplete data. The method not only improves the regression accuracy, but also enhances the efficiency of symbolic regression on high-dimensional incomplete data. Another challenging problem is data shortage. This issue is even more challenging when the data is incomplete. To handle this situation, this thesis develops transfer learning methods to improve symbolic regression in domains with incomplete and limited data. These methods utilise two powerful abilities of GP: feature construction and feature selection. The results show the ability of these methods to achieve positive transfer learning from domains with complete data to different (but related) domains with incomplete data. In summary, the thesis develops a range of approaches to improving the effectiveness and efficiency of symbolic regression on incomplete data by developing a number of GP-based methods. The methods are evaluated using different types of data sets considering various missingness and learning scenarios.

show abstract

A parallel and distributed semantic Genetic Programming system

Cited by 3 publications

References 17 publications

Pruning Techniques for Mixed Ensembles of Genetic Programming Models

Pruning Techniques for Mixed Ensembles of Genetic Programming Models

Genetic Programming for Symbolic Regression on Incomplete Data

Genetic Programming for Symbolic Regression on Incomplete Data

Contact Info

Product

Resources

About