Bounding bloat in genetic programming

Abstract

While many optimization problems work with a fixed number of decision variables and thus a fixed-length representation of possible solutions, genetic programming (GP) works on variable-length representations. A naturally occurring problem is that of bloat (unnecessary growth of solutions) slowing down optimization. Theoretical analyses could so far not bound bloat and required explicit assumptions on the magnitude of bloat.In this paper we analyze bloat in mutation-based genetic programming for the two test functions Order and Majority. We overcome previous assumptions on the magnitude of bloat and give matching or close-to-matching upper and lower bounds for the expected optimization time.In particular, we show that the (1+1) GP takes (i) ΘpT init`n log nq iterations with bloat control on Order as well as Majority; and (ii) OpT init log T init`n plog nq 3 q and ΩpT init`n log nq (and ΩpT init log T init q for n " 1) iterations without bloat control on Majority. 1

Computational Complexity Analysis of Genetic Programming

“…A tight bound for the (1 + 1) GP, showing that the larger Poisson mutations do not affect the asymptotic runtime, has recently been proven [6], confirming a previous conjecture [53].…”

“…In many cases, an analysis that provides positive runtime results is only made tractable because "the fitness structure of the model problems is simple, and the algorithms use only a simple hierarchical variable length mutation operator" [9]. In particular, variable-length representations often complicate the analysis of GP systems, and require "rather deep insights into the optimization process and the growth of the GP-trees" [6].…”

“…The performance of the (1 + 1) GP with lexicographic parsimony pressure on Order has been considered by Nguyen et al [38] and Doerr et al [6]. This mechanism controls bloat by preferring trees of smaller size when ties amongst solutions of equal fitness are broken.…”

The linear hidden subset problem for the (1 + 1) EA with scheduled and adaptive mutation rates

“…Theorem 6 (Multiplicative drift, lower bound [15]). Let (X t ) t≥0 be random variables describing a Markov process over a finite state space S ⊂ R + .…”

“…It is clear that this bound is not true for all n, for example, if p t = 1/m for all t, then E[Z m ] = β ′ /e · m ln m, which is larger than m ln m if β ′ > e. In order to show that the bound holds for infinitely many n, we consider instead a weighted average B = n ρ(n)Z n over many n. The choice of ρ is delicate, but it turns out that ρ(n) = 1/n 2 is the right choice. 15 In the technical part of the proof, we derive an upper bound on B = n Z n /n 2 . Note that Z n counts the single bit flips until time T ′ n .…”