Cost-benefit analysis of using heuristics in ACGP

Abstract

Abstract-Constrained Genetic Programming (CGP) is a method of searching the Genetic Programming search space non-uniformly, giving preferences to certain subspaces according to some heuristics. Adaptable CGP (ACGP) is a method for discovery of the heuristics. CGP and ACGP have previously demonstrated their capabilities using first-order heuristics: parent-child probabilities. Recently, the same advantage has been shown for second-order heuristics: parent-children probabilities. A natural question to ask is whether we can benefit from extending ACGP with deeperorder heuristics. This paper attempts to answer this question by performing cost-benefit analysis while simulating the higher-order heuristics environment. We show that this method cannot be extended beyond the current second or possibly third-order heuristics without a new method to deal with the sheer number of such deeper-order heuristics.Keywords-Genetic Programming, Adaptable Constrained Genetic Programming, Building Block Hypothesis, Heuristic. I. BACKGROUNDGenetic Programming (GP) is an evolutionary computation method merging concepts from computer science and biology. It has been shown to provide robust solutions for problems such as evolving computer programs, designing logic circuits, discovering mathematical equations, and solving other optimization and combinatorial problems where other solutions are not practical or unknown [2].Genetic Programming represents a population of candidate solutions as trees, lists or graphs, but the dominant representation, and the one this paper concentrates on, is the tree representation [2]. These trees are composed of elements from a predetermined set of functions and terminals, called here labels. A given function can label any node whose number of subtrees equals the function arity. Terminal labels, such as constants or variables from the problem environment, label the leaves.The search space is the space of all (up to some size limit) trees that can be labeled with the provided labels. Somewhere in the space, we should find the actual solution to the problem at hand or otherwise GP will not be capable of ever finding the solution. The quality of a single solution-tree, and therefore its "proximity" to the sought solution, is the tree's evaluation through a provided black-box fitness function.After the initial sampling in the GP population, new solutions are evolved using genetic operators such as crossover and mutation, while guided by selection based on fitness evaluations. In crossover, randomly chosen sub-trees are exchanged between two parents. In mutation, a randomly chosen sub-tree is regrown. One critical issue with GP design is the choice of the labels. The labels, along with limitations on explored tree sizes, determine the search space for GP (while population, selection and operators determine the means of searching the space). If the label set is not sufficient, the desired solution does not exist in the search space and therefore GP will never find it. This is often referred to as the sufficiency...