Design and Analysis of Optimization Algorithms Using Computational Statistics

Bartz-Beielstein, Thomas; Vrahatis, Michael N.

doi:10.1002/anac.200410007

Cited by 65 publications

(57 citation statements)

References 53 publications

(83 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the context of parameter optimization, EGO could be used to optimize deterministic algorithms with continuous parameters on a single problem instance. Two independent lines of work extended EGO to noisy functions, which in the context of parameter optimization, allow the consideration of randomized algorithms: the sequential Kriging optimization (SKO) algorithm by Huang et al (2006), and the sequential parameter optimization (SPO) procedure by Bartz-Beielstein et al(2004b;.…”

Section: Introductionmentioning

confidence: 99%

Sequential Model-Based Parameter Optimization: an Experimental Investigation of Automated and Interactive Approaches

Hutter

Bartz-Beielstein

Hoos

et al. 2010

Experimental Methods for the Analysis of Optimization Algorithms

View full text Add to dashboard Cite

This work experimentally investigates model-based approaches for optimizing the performance of parameterized randomized algorithms. Such approaches build a response surface model and use this model for finding good parameter settings of the given algorithm. We evaluated two methods from the literature that are based on Gaussian process models: sequential parameter optimization (SPO) (Bartz-Beielstein et al, 2005) and sequential Kriging optimization (SKO) (Huang et al, 2006). SPO performed better "out-of-the-box," whereas SKO was competitive when response values were log transformed. We then investigated key design decisions within the SPO paradigm, characterizing the performance consequences of each. Based on these findings, we propose a new version of SPO, dubbed SPO + , which extends SPO with a novel intensification procedure and a log-transformed objective function. In a domain for which performance results for other (model-free) parameter optimization approaches are available, we demonstrate that SPO + achieves state-of-the-art performance. Finally, we compare this automated parameter tuning approach to an interactive, manual process that makes use of classical regression techniques. This interactive approach is particularly useful when only a relatively small number of parameter configurations can be evaluated. Because it can relatively quickly draw attention to important parameters and parameter interactions, it can help experts gain insights into the parameter response of a given algorithm and identify reasonable parameter settings.

show abstract

Section: Introductionmentioning

confidence: 99%

Sequential Model-Based Parameter Optimization: an Experimental Investigation of Automated and Interactive Approaches

Hutter

Bartz-Beielstein

Hoos

et al. 2010

Experimental Methods for the Analysis of Optimization Algorithms

View full text Add to dashboard Cite

show abstract

“…The most promising points are then added to the population. Although in [3], [2] regression models are used to model the utilities, succeded by stochastic models [4], it is in principle a general framework suited for a large range of modeling techniques.…”

Section: Sequential Parameter Optimizationmentioning

confidence: 99%

“…Sharpening has not been introduced before as a separate technique for testing, although it has been used previously, inside the SPOmethod by Bartz-Beielstein et al [2]. Thus, in this paper we do not invent it, but designate it as an independent add-on, and give it the name sharpening.…”

Section: B Sharpeningmentioning

confidence: 99%

“…For the top layer, we have tested three different algorithms combined with the two additional add-ons namely: [2] and Racing [14], are open source and freely available from the websites of the corresponding authors. The additional addons are handcrafted changes to the three algorithms and their exact implementations are algorithm specific.…”

Section: Combining Racing and Sharpeningmentioning

confidence: 99%

See 1 more Smart Citation

Comparing parameter tuning methods for evolutionary algorithms

Smit

Eiben

2009

2009 IEEE Congress on Evolutionary Computation

190

141

View full text Add to dashboard Cite

Abstract-Tuning the parameters of an evolutionary algorithm (EA) to a given problem at hand is essential for good algorithm performance. Optimizing parameter values is, however, a non-trivial problem, beyond the limits of human problem solving.In this light it is odd that no parameter tuning algorithms are used widely in evolutionary computing. This paper is meant to be stepping stone towards a better practice by discussing the most important issues related to tuning EA parameters, describing a number of existing tuning methods, and presenting a modest experimental comparison among them. The paper is concluded by suggestions for future researchhopefully inspiring fellow researchers for further work.Index Terms-evolutionary algorithms, parameter tuning I. BACKGROUND AND OBJECTIVESEvolutionary Algorithms (EA) form a rich class of stochastic search methods that share the basic principles of incrementally improving the quality of a set of candidate solutions by means of variation and selection [7], [5]. Algorithms in this class are all based on the same generic framework whose details need to be specified to obtain a particular EA. It is customary to call these details EA parameters, and designing an EA for a given application amounts to selecting good values for these parameters.Setting EA parameters is commonly divided into two cases, parameter tuning and parameter control [6]. In case of parameter control the parameter values are changing during an EA run. In this case one needs initial parameter values and suitable control strategies, that in turn can be deterministic, adaptive, or self-adaptive. Parameter tuning is easier in that the parameter values are not changing during a run, hence only a single value per parameter is required. Nevertheless, even the problem of tuning an EA for a given application is hard because there is a large number of options, but only little knowledge about the effect of EA parameters on EA performance. EA users mostly rely on conventions (mutation rate should be low), ad hoc choices (why not use uniform crossover), and experimental comparisons on a limited scale (testing combinations of three different crossover rates and three different mutation rates).The main objective of this paper is to illustrate the feasibility of using tuning algorithms, thereby motivating their usage. To this end, we describe three different approaches to algorithmic parameter tuning (meta-EA, meta-EDA, SPO) and show their (dis)advantages when tuning EA parameters for solving the Rastrigin function. While the limited scale (one single fitness landscape and one algorithm to be tuned) prevents general conclusions, we do obtain a convincing Vrije Universiteit Amsterdam, The Netherlands, {sksmit, gusz}@cs.vu.nl showcase and some very interesting insights whose generalization requires much more experimental research. II. PARAMETERS, TUNERS, AND UTILITY LANDSCAPESIntuitively, there is a difference between choosing a good crossover operator and choosing a good value for the related crossover rate p c . This differ...

show abstract

“…State-of-the-art model-based approaches use Gaussian stochastic processes (also known as Kriging models) to fit a response surface model. Two independent lines of work extended Kriging to noisy functions, which in the context of parameter optimization, allow the consideration of randomized algorithms: the sequential Kriging optimization (SKO) algorithm by Huang et al (2006), and the sequential parameter optimization (SPO) procedure by Bartz-Beielstein et al (2004. present a comprehensive overview on (off-line) parameter tuning.…”

Section: Tuningmentioning

confidence: 99%

An Overview on Evolutionary Algorithms

Data Mining Using Grammar Based Genetic Programming and Applications

View full text Add to dashboard Cite

Design and Analysis of Optimization Algorithms Using Computational Statistics

Cited by 65 publications

References 53 publications

Sequential Model-Based Parameter Optimization: an Experimental Investigation of Automated and Interactive Approaches

Sequential Model-Based Parameter Optimization: an Experimental Investigation of Automated and Interactive Approaches

Comparing parameter tuning methods for evolutionary algorithms

An Overview on Evolutionary Algorithms

Contact Info

Product

Resources

About