Pflacco: Feature-Based Landscape Analysis of Continuous and
                    Constrained Optimization Problems in Python

Prager, Raphael Patrick; Trautmann, Heike

doi:10.1162/evco_a_00341

Cited by 8 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…c 84 ) is used to categorize the test problems for hyperparameter prediction. The vector C includes 41 features evaluated by the pFlacco package [23] and 43 features based on VM [12]. Only ELA features that are based on a fixed sample of points are used since additional objective evaluations are not permissible with a fixed computational budget.…”

Section: General Setupmentioning

confidence: 99%

Exploratory Landscape Validation for Bayesian Optimization Algorithms

Agasiev,

Karpenko

2024

Mathematics

View full text Add to dashboard Cite

Bayesian optimization algorithms are widely used for solving problems with a high computational complexity in terms of objective function evaluation. The efficiency of Bayesian optimization is strongly dependent on the quality of the surrogate models of an objective function, which are built and refined at each iteration. The quality of surrogate models, and hence the performance of an optimization algorithm, can be greatly improved by selecting the appropriate hyperparameter values of the approximation algorithm. The common approach to finding good hyperparameter values for each iteration of Bayesian optimization is to build surrogate models with different hyperparameter values and choose the best one based on some estimation of the approximation error, for example, a cross-validation score. Building multiple surrogate models for each iteration of Bayesian optimization is computationally demanding and significantly increases the time required to solve an optimization problem. This paper suggests a new approach, called exploratory landscape validation, to find good hyperparameter values with less computational effort. Exploratory landscape validation metrics can be used to predict the best hyperparameter values, which can improve both the quality of the solutions found by Bayesian optimization and the time needed to solve problems.

show abstract

Section: General Setupmentioning

confidence: 99%

Exploratory Landscape Validation for Bayesian Optimization Algorithms

Agasiev,

Karpenko

2024

Mathematics

View full text Add to dashboard Cite

show abstract

“…To obtain the AOCC over multiple runs, we simply take the average. For the landscape analysis, we make use of the pFlacco [26] package to calculate the ELA features. We use a sample size of 1 000𝑑 points, sampled using a Sobol' sequence.…”

Section: Setupmentioning

confidence: 99%

Using Affine Combinations of BBOB Problems for Performance Assessment

Vermetten

Doerr

2023

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Choosing a set of benchmark problems is often a key component of any empirical evaluation of iterative optimization heuristics. In continuous, single-objective optimization, several sets of problems have become widespread, including the well-established BBOB suite. While this suite is designed to enable rigorous benchmarking, it is also commonly used for testing methods such as algorithm selection, which the suite was never designed around.We present the MA-BBOB function generator, which uses the BBOB suite as component functions in an affine combination. In this work, we describe the full procedure to create these affine combinations and highlight the trade-offs of several design decisions, specifically the choice to place the optimum uniformly at random in the domain. We then illustrate how this generator can be used to gain more low-level insight into the function landscapes through the use of exploratory landscape analysis.Finally, we show a potential use-case of MA-BBOB in generating a wide set of training and testing data for algorithm selectors.Using this setup, we show that the basic scheme of using a set of landscape features to predict the best algorithm does not lead to optimal results, and that an algorithm selector trained purely on the BBOB functions generalizes poorly to the affine combinations.CCS Concepts: • Theory of computation → Random search heuristics; Design and analysis of algorithms; Bio-inspired optimization.

show abstract

“…Later studies have further augmented the set of available low-level feature sets. Two packages, namely flacco [17] and pflacco [18], provide an implementation of these features in the programming languages R and Python respectively.…”

Section: Exploratory Landscape Analysismentioning

confidence: 99%

“…To enable our AAS model, we generate the relevant ELA features (cf. Section III) using the Python package pflacco [18]. One of our earlier intuitions was that the cardinality of each categorical variable of an MVP might have an influence on the chosen encoding strategy.…”

Section: Exploratory Landscape Analysis Feature Generationmentioning

confidence: 99%

Investigating the Viability of Existing Exploratory Landscape Analysis Features for Mixed-Integer Problems

Prager

Trautmann

2023

Proceedings of the Companion Conference on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

Exploratory landscape analysis and fitness landscape analysis in general have been pivotal in facilitating problem understanding, algorithm design and endeavors such as automated algorithm selection and configuration. These techniques have largely been limited to search spaces of a single domain. In this work, we provide the means to compute exploratory landscape features for mixed-variable problems where the decision space is a mixture of continuous, binary, integer, and categorical variables.This is achieved by utilizing existing encoding techniques originating from machine learning. We provide a comprehensive juxtaposition of the results based on these different techniques. To further highlight their merit for practical applications, we design and conduct an automated algorithm selection study based on a hyperparameter optimization benchmark suite.We derive a meaningful compartmentalization of these benchmark problems by clustering based on the used landscape features. The identified clusters mimic the behavior the used algorithms exhibit. Meaning, the different clusters have different best performing algorithms. Finally, our trained algorithm selector is able to close the gap between the single best and the virtual best solver by 57.5% over all benchmark problems.

show abstract

Pflacco: Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems in Python

Cited by 8 publications

References 0 publications

Exploratory Landscape Validation for Bayesian Optimization Algorithms

Exploratory Landscape Validation for Bayesian Optimization Algorithms

Using Affine Combinations of BBOB Problems for Performance Assessment

Investigating the Viability of Existing Exploratory Landscape Analysis Features for Mixed-Integer Problems

Contact Info

Product

Resources

About