Efficient relevance estimation and value calibration of evolutionary algorithm parameters

Nannen, Volker; Eiben, A. E.

doi:10.1109/cec.2007.4424460

Cited by 154 publications

(133 citation statements)

References 12 publications

Supporting

Mentioning

124

Contrasting

Unclassified

Order By: Relevance

“…Notable exceptions in the literature include experimental design based on linear models [11,12], an entropy-based measure [13], and visualization methods for interactive parameter exploration, such as contour plots [14]. However, to the best of our knowledge, none of these methods has so far been applied to study the configuration spaces of state-of-the-art highly parametric solvers; their applicability is unclear due to the high dimensionality of these spaces and the prominence of discrete parameters (which, e.g., linear models cannot handle gracefully).…”

Section: Introductionmentioning

confidence: 99%

Identifying Key Algorithm Parameters and Instance Features Using Forward Selection

Hutter

Hoos

Leyton‐Brown

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Most state-of-the-art algorithms for large scale optimization expose free parameters, giving rise to combinatorial spaces of possible configurations. Typically, these spaces are hard for humans to understand. In this work, we study a model-based approach for identifying a small set of both algorithm parameters and instance features that suffices for predicting empirical algorithm performance well. Our empirical analyses on a wide variety of hard combinatorial problem benchmarks (spanning SAT, MIP, and TSP) show that-for parameter configurations sampled uniformly at random-very good performance predictions can typically be obtained based on just two key parameters, and that similarly, few instance features and algorithm parameters suffice to predict the most salient algorithm performance characteristics in the combined configuration/feature space. We also use these models to identify settings of these key parameters that are predicted to achieve the best overall performance, both on average across instances and in an instance-specific way. This serves as a further way of evaluating model quality and also provides a tool for further understanding the parameter space. We provide software for carrying out this analysis on arbitrary problem domains and hope that it will help algorithm developers gain insights into the key parameters of their algorithms, the key features of their instances, and their interactions.

show abstract

Section: Introductionmentioning

confidence: 99%

Identifying Key Algorithm Parameters and Instance Features Using Forward Selection

Hutter

Hoos

Leyton‐Brown

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Nannen and Eiben have introduced a method for Relevance Estimation and Value Calibration of parameters (REVAC) in [16], [15]. Although the REVAC method was not designed with Estimation of Distribution Algorithms (EDA) in mind, it is based on the same general idea [10].…”

Section: B Meta Estimation Of Distribution Algorithmmentioning

confidence: 99%

Comparing parameter tuning methods for evolutionary algorithms

Smit

Eiben

2009

2009 IEEE Congress on Evolutionary Computation

190

141

View full text Add to dashboard Cite

Abstract-Tuning the parameters of an evolutionary algorithm (EA) to a given problem at hand is essential for good algorithm performance. Optimizing parameter values is, however, a non-trivial problem, beyond the limits of human problem solving.In this light it is odd that no parameter tuning algorithms are used widely in evolutionary computing. This paper is meant to be stepping stone towards a better practice by discussing the most important issues related to tuning EA parameters, describing a number of existing tuning methods, and presenting a modest experimental comparison among them. The paper is concluded by suggestions for future researchhopefully inspiring fellow researchers for further work.Index Terms-evolutionary algorithms, parameter tuning I. BACKGROUND AND OBJECTIVESEvolutionary Algorithms (EA) form a rich class of stochastic search methods that share the basic principles of incrementally improving the quality of a set of candidate solutions by means of variation and selection [7], [5]. Algorithms in this class are all based on the same generic framework whose details need to be specified to obtain a particular EA. It is customary to call these details EA parameters, and designing an EA for a given application amounts to selecting good values for these parameters.Setting EA parameters is commonly divided into two cases, parameter tuning and parameter control [6]. In case of parameter control the parameter values are changing during an EA run. In this case one needs initial parameter values and suitable control strategies, that in turn can be deterministic, adaptive, or self-adaptive. Parameter tuning is easier in that the parameter values are not changing during a run, hence only a single value per parameter is required. Nevertheless, even the problem of tuning an EA for a given application is hard because there is a large number of options, but only little knowledge about the effect of EA parameters on EA performance. EA users mostly rely on conventions (mutation rate should be low), ad hoc choices (why not use uniform crossover), and experimental comparisons on a limited scale (testing combinations of three different crossover rates and three different mutation rates).The main objective of this paper is to illustrate the feasibility of using tuning algorithms, thereby motivating their usage. To this end, we describe three different approaches to algorithmic parameter tuning (meta-EA, meta-EDA, SPO) and show their (dis)advantages when tuning EA parameters for solving the Rastrigin function. While the limited scale (one single fitness landscape and one algorithm to be tuned) prevents general conclusions, we do obtain a convincing Vrije Universiteit Amsterdam, The Netherlands, {sksmit, gusz}@cs.vu.nl showcase and some very interesting insights whose generalization requires much more experimental research. II. PARAMETERS, TUNERS, AND UTILITY LANDSCAPESIntuitively, there is a difference between choosing a good crossover operator and choosing a good value for the related crossover rate p c . This differ...

show abstract

“…This category mostly includes statistical methods derived from Design Of Experiments (see e.g., [11][12][13][14]). Although more efficient than standard ANOVA methods, Parameter Tuning relies on extensive, computationally expensive, experiments.…”

Section: Parameter Setting In Easmentioning

confidence: 99%

Dynamic Multi-Armed Bandits and Extreme Value-Based Rewards for Adaptive Operator Selection in Evolutionary Algorithms

Fialho

Costa

Schoenauer

et al. 2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The performance of many efficient algorithms critically depends on the tuning of their parameters, which on turn depends on the problem at hand. For example, the performance of Evolutionary Algorithms critically depends on the judicious setting of the operator rates. The Adaptive Operator Selection (AOS) heuristic that is proposed here rewards each operator based on the extreme value of the fitness improvement lately incurred by this operator, and uses a Multi-Armed Bandit (MAB) selection process based on those rewards to choose which operator to apply next. This Extreme-based Multi-Armed Bandit approach is experimentally validated against the Average-based MAB method, and is shown to outperform previously published methods, whether using a classical Average-based rewarding technique or the same Extreme-based mechanism. The validation test suite includes the easy One-Max problem and a family of hard problems known as "Long k-paths".

show abstract

Efficient relevance estimation and value calibration of evolutionary algorithm parameters

Cited by 154 publications

References 12 publications

Identifying Key Algorithm Parameters and Instance Features Using Forward Selection

Identifying Key Algorithm Parameters and Instance Features Using Forward Selection

Comparing parameter tuning methods for evolutionary algorithms

Dynamic Multi-Armed Bandits and Extreme Value-Based Rewards for Adaptive Operator Selection in Evolutionary Algorithms

Contact Info

Product

Resources

About