General Lower Bounds for Evolutionary Algorithms

Abstract

Abstract.@inProceedings{lbes, author={O. Teytaud and S. Gelly}, title={General lower bounds for evolutionary algorithms}, booktitle = {$10^{th}$ International Conference on Parallel ProblemSolving from Nature (PPSN 2006), 10 pages,}, year=2006}Evolutionary optimization, among which genetic optimization, is a general framework for optimization. It is known (i) easy to use (ii) robust (iii) derivative-free (iv) unfortunately slow. Recent work [8] in particular show that the convergence rate of some widely used evolution strategies (evolutionary optimization for continuous domains) can not be faster than linear (i.e. the logarithm of the distance to the optimum can not decrease faster than linearly), and that the constant in the linear convergence (i.e. the constant C such that the distance to the optimum after n steps is upper bounded by C n ) unfortunately converges quickly to 1 as the dimension increases to ∞. We here show a very wide generalization of this result: all comparison-based algorithms have such a limitation. Note that our result also concerns methods like the Hooke & Jeeves algorithm, the simplex method, or any direct search method that only compares the values to previously seen values of the fitness. But it does not cover methods that use the value of the fitness (see [5] for cases in which the fitness-values are used), even if these methods do not use gradients. The former results deal with convergence with respect to the number of comparisons performed, and also include a very wide family of algorithms with respect to the number of function-evaluations. However, there is still place for faster convergence rates, for more original algorithms using the full ranking information of the population and not only selections among the population. We prove that, at least in some particular cases, using the full ranking information can improve these lower bounds, and ultimately provide superlinear convergence results.

Lower Bounds for Comparison Based Evolution Strategies Using VC-dimension and Sign Patterns

“…The optimality of this comparisonbased principle for some robustness criterion was shown in [11] -see also [4,29,5]. The tools provided in [26] for proving lower bounds for evolutionary algorithms are interesting, but, as pointed out by the authors, the bounds for the (µ, λ)-ES are far too small for µ > 1 and λ larger than the dimension, while the discrete case provides essentially trivial results. In this work, we present improved lower bounds on the convergence rate of evolution strategies of type (µ + , λ)-ES in terms of the VC-dimension of level sets of the fitness functions.…”

“…This fact has been used in [26] in order to provide lower bounds that match some upper bounds known for evolutionary algorithms [9,2,23]. The optimality of this comparisonbased principle for some robustness criterion was shown in [11] -see also [4,29,5].…”

“…As explained above, the present work builds on [26], where the branching factor was introduced for the analysis of evolutionary algorithms; results obtained here are improved in the case of families of fitness functions with bounded VC-dimension. The first lower bounds for some evolutionary algorithms in continuous domains were provided in [17,15,16].…”

“…We also generalize existing results in the discrete case by considering arbitrary values of parameters λ and µ, and improve previous results from [10] in the special cases of (1 + λ)-ES or (µ + 1)-ES -a detailed comparison with state of the art is provided within the paper regarding these cases. We also remark that [25,Theorem 2], which is a complexity lower bound for a variant of particle swarm optimization, is included in the main theorem of [26].…”

OneMax in Black-Box Models with Several Restrictions

“…Another important bound for the OneMax problem is the simple Θ(n) bound for comparison-based algorithms as introduced in [TG06]. 2 Since (1+1) memory-restricted ranking-based algorithms are comparison-based, this gives a linear lower bound for their complexity on OneMax.…”