Improving genetic programming based symbolic regression using deterministic machine learning

Abstract

Abstract-Symbolic regression (SR) is a well studied method in genetic programming (GP) for discovering free-form mathematical models from observed data. However, it has not been widely accepted as a standard data science tool. The reluctance is in part due to the hard to analyze random nature of GP and scalability issues. On the other hand, most popular deterministic regression algorithms were designed to generate linear models and therefore lack the flexibility of GP based SR (GP-SR). Our hypothesis is that hybridizing these two techniques will create a synergy between the GP-SR and deterministic approaches to machine learning, which might help bring the GP based techniques closer to the realm of big learning. In this paper, we show that a hybrid deterministic/GP-SR algorithm outperforms GP-SR alone and the state-of-the-art deterministic regression technique alone on a set of multivariate polynomial symbolic regression tasks as the system to be modeled becomes more multivariate.

A Deterministic and Symbolic Regression Hybrid Applied to Resting-State fMRI Data

“…By taking advantage of the state-of-the-art deterministic regression methods, our algorithm aims to ease the burden of feature extraction on GP-SR and help it excel in model building, especially when higher order interactions between the variables exist. We have shown on a synthetic data suite with up to fourth order variable interactions that our technique significantly improves the performance of GP-SR as the data dimensionality increases ( Icke and Bongard (2013)). …”

Transformation of input space using statistical moments: EA-based approach

“…The authors verified FFX on a broad set of realworld problems with different numbers of variables ranging from 13 to 1468. Later, Icke and Bongard [10] hybridised FFX and GP to create an improved learner for symbolic regression problems. In their work, the authors showed that a hybrid deterministic/GP for symbolic regression outperforms GP alone and several state-of-the-art deterministic regression techniques alone on a set of multivariate polynomial symbolic regression tasks.…”

Generalisation Enhancement via Input Space Transformation: A GP Approach

“…Later, Icke and Bongard [5] hybridised FFX and GP to create an improved learner for symbolic regression problems. In this work, the authors showed that a hybrid deterministic/GP for symbolic regression outperforms GP alone and several stateof-the-art deterministic regression techniques alone on a set of multivariate polynomial symbolic regression tasks.…”

“…The total number of test problems is 20 (i.e., 5 test functions ⇥4 different variables sizes). For all test problems, we randomly generated three disjoint sets; a training set of 100 points, a validation set of 50 points, and a testing set of 150 points from the interval [ 5,5]. All techniques have been compared based on the average of absolute errors on the testing set.…”