Feature Based Algorithm Configuration: A Case Study with Differential Evolution

Belkhir, Nacim; Dréo, Johann; Savéant, Pierre; Schoenauer, Marc

doi:10.1007/978-3-319-45823-6_15

Cited by 24 publications

(34 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are a number of avenues to investigate, discussed above, which may improve the methodology further. In particular an investigation into more sophisticated function features, such as those used in FBAC [3], is a high priority. The long term goal should be to extend this methodology to automatically select the most appropriate evolutionary algorithm for a problem, not just the control parameters.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Predicting effective control parameters for differential evolution using cluster analysis of objective function features

Walton

Brown

2019

J Heuristics

View full text Add to dashboard Cite

A methodology is introduced which uses three simple objective function features to predict effective control parameters for differential evolution. This is achieved using cluster analysis techniques to classify objective functions using these features. Information on prior performance of various control parameters for each classification is then used to determine which control parameters to use in future optimisations. Our approach is compared to an state-of-the-art adaptive technique along with non-adaptive techniques. Two accepted bench mark suites are used to compare performance, in all cases we show that the improvement resulting from our approach is statistically significant. The majority of the computational effort of this methodology is performed off-line, however even taking into account the additional on-line cost our approach outperforms other adaptive techniques. We also study the key tuning parameters of our methodology, which further support the finding that the simple features selected are predictors of effective 1 arXiv:1806.09432v1 [cs.NE] 25 Jun 2018 control parameters. The findings presented in this paper are significant because they show that simple to calculate features of objective functions can help select control parameters for optimisation algorithms. This can have an immediate positive impact the application of these optimisation algorithms on real world problems where it is often difficult to select effective control parameters.

show abstract

Section: Discussionmentioning

confidence: 99%

“…There are a number of metrics which can be utilised to define the performance of an optimisation algorithm [7,3]. The meaning of performance may change depending on the application [13], but in general we wish to reduce the objective function value with a small number of objective function evaluations.…”

Section: Optimisation Algorithm Performance Metricmentioning

confidence: 99%

Predicting effective control parameters for differential evolution using cluster analysis of objective function features

Walton

Brown

2019

J Heuristics

View full text Add to dashboard Cite

show abstract

“…The per-instance variant of the algorithm configuration problem, which can be seen as a generalisation of per-instance algorithm selection, largely remains an open challenge (see, e.g., Hutter et al, 2006;Belkhir et al, 2016Belkhir et al, , 2017, and we briefly discuss it further in Section 6.…”

Section: Algorithm Selection and Related Problemsmentioning

confidence: 99%

“…Note that by using a platform-independent web-application of the flacco package 1 (Hanster and Kerschke, 2017), researchers and practitioners, who are unfamiliar with R, can also benefit from this extensive collection of more than 300 landscape features. Belkhir et al (2016Belkhir et al ( , 2017 were among the first to leverage the ELA features provided by flacco for per-instance algorithm configuration.…”

Section: Continuous Problemsmentioning

confidence: 99%

Automated Algorithm Selection: Survey and Perspectives

Kerschke

Hoos

Neumann

et al. 2019

Evolutionary Computation

307

184

View full text Add to dashboard Cite

It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different algorithms. This is particularly pronounced for computationally hard problems, where in most cases, no single algorithm defines the state of the art; instead, there is a set of algorithms with complementary strengths. This performance complementarity can be exploited in various ways, one of which is based on the idea of selecting, from a set of given algorithms, for each problem instance to be solved the one expected to perform best. The task of automatically selecting an algorithm from a given set is known as the per-instance algorithm selection problem and has been intensely studied over the past 15 years, leading to major improvements in the state of the art in solving a growing number of discrete combinatorial problems, including propositional satisfiability and AI planning. Perinstance algorithm selection also shows much promise for boosting performance in solving continuous and mixed discrete/continuous optimisation problems.This survey provides an overview of research in automated algorithm selection, ranging from early and seminal works to recent and promising application areas. Different from earlier work, it covers applications to discrete and continuous problems, and discusses algorithm selection in context with conceptually related approaches, such as algorithm configuration, scheduling or portfolio selection. Since informative and cheaply computable problem instance features provide the basis for effective perinstance algorithm selection systems, we also provide an overview of such features for discrete and continuous problems. Finally, we provide perspectives on future work in the area and discuss a number of open research challenges.

show abstract

“…In contrast to [18], Muñoz et al [26], the empirical performance model was learned by using a multi-layer neural network method. More recently, based on aforementionned features in Section 4, Belkhir et al [5] investigated the PIAC on Differential Evolution [29], by considering different sets of features and a discretization of the parameter space (≈ 8000 parameter configurations. According to [18], an empirical performance model is learned with a Random Forest regression, and based on a cross-validation procedure where each test function of the BBOB test were removed one at a time, Belkhir et al [5] empirically demonstrated that predicted parameter setting found with PIAC can outperform a robust parameter setting, and approach the best parameter setting, whereas remains outperformed by a specialized parameter setting found with an AC method like SMAC.…”

Section: Piac In Continuous Domainmentioning

confidence: 99%

Per instance algorithm configuration of CMA-ES with limited budget

Belkhir

Dréo

Savéant

et al. 2017

Proceedings of the Genetic and Evolutionary Computation Conference

Self Cite

View full text Add to dashboard Cite

Per Instance Algorithm Configuration (PIAC) relies on features that describe problem instances. It builds an Empirical Performance Model (EPM) from a training set made of (instance, parameter configuration) pairs together with the corresponding performance of the algorithm at hand. This paper presents a case study in the continuous black-box optimization domain, using features proposed in the literature. The target algorithm is CMA-ES, and three of its hyper-parameters. Special care is taken to the computational cost of the features. The EPM is learned on the BBOB benchmark, but tested on independent test functions gathered from the optimization literature.The results demonstrate that the proposed approach can outperform the default setting of CMA-ES with as few as 30 or 50 time the problem dimension additional function evaluations for feature computation.

show abstract

Feature Based Algorithm Configuration: A Case Study with Differential Evolution

Cited by 24 publications

References 17 publications

Predicting effective control parameters for differential evolution using cluster analysis of objective function features

Predicting effective control parameters for differential evolution using cluster analysis of objective function features

Automated Algorithm Selection: Survey and Perspectives

Per instance algorithm configuration of CMA-ES with limited budget

Contact Info

Product

Resources

About