Geometric Semantic Genetic Programming with Local Search

Abstract

Since its introduction, Geometric Semantic Genetic Programming (GSGP) has aroused the interest of numerous researchers and several studies have demonstrated that GSGP is able to effectively optimize training data by means of small variation steps, that also have the effect of limiting overfitting. In order to speed up the search process, in this paper we propose a system that integrates a local search strategy into GSGP (called GSGP-LS). Furthermore, we present a hybrid approach, that combines GSGP and GSGP-LS, aimed at exploiting both the optimization speed of GSGP-LS and the ability to limit overfitting of GSGP. The experimental results we present, performed on a set of complex real-life applications, show that GSGP-LS achieves the best training fitness while converging very quickly, but severely overfits. On the other hand, GSGP converges slowly relative to the other methods, but is basically not affected by overfitting. The best overall results were achieved with the hybrid approach, allowing the search to converge quickly, while also exhibiting a noteworthy ability to limit overfitting. These results are encouraging, and suggest that future GSGP algorithms should focus on finding the correct balance between the greedy optimization of a local search strategy and the more robust geometric semantic operators.

Comparison of semantic-based local search methods for multiobjective genetic programming

“…This case study resulted in good model accuracy with a speededup search process. In order to accelerate convergence, Castelli et al [8] proposed a hybrid algorithm combining GSGP and the above method. The results show this hybrid method allows the search to converge quickly while also exhibiting a noteworthy ability to limit overfitting.…”

Analysing symbolic regression benchmarks under a meta-learning approach

“…We started by compiling a list of 80 datasets employed in 26 papers working with symbolic regression problems published at GECCO from 2013 to 2017. From these 80 datasets, we could not nd the description of two synthetic datasets-Sext [12] and Nguyen-12 [12,16]-and we could not nd seven real-world datasets available on line-Dow chemical [28], Plasma protein binding level (PPB) [5,9,29], Tower data [14,15,29], NOX [1,2], Wind (WND) [15], Median lethal dose (toxicity/LD50) [5,9] and Human oral bioavailability [5,8,9,29] 1 .…”

A Dispersion Operator for Geometric Semantic Genetic Programming

“…Despite the unimodal fitness landscape, the randomness present in these operators, available in the form of random real functions or constants, has been shown to be a better way to explore the space, in terms of generalisation, when compared to modifications of these operators where the randomness is replaced by decisions based on the training error [1,6,9]. Definition 3.…”

“…Given the arrays countgt and count lt , it iterates through the dimensions calculating the inequalities. The method isolates m (line 4) and stores the right side of the inequality along with its type ('lb' and 'ub' for lower and upper bounds, respectively) in the set B (lines [5][6][7][8][9][10][11][12][13][14]. Notice that when s(p)[i] < 0, we have to invert the inequality sign when m is isolated.…”

“…This fact justifies the use of other operators (crossover and mutation) in order to further explore the search space. Although the location of the optimal solution is known, making decisions based only on the distance to the solution (equivalent to the training error) may lead to overfitting, as presented in other works [1,6,9].…”