Personalizing performance regression models to black-box optimization problems

Abstract

Accurately predicting the performance of different optimization algorithms for previously unseen problem instances is crucial for high-performing algorithm selection and configuration techniques. In the context of numerical optimization, supervised regression approaches built on top of exploratory landscape analysis are becoming very popular. From the point of view of Machine Learning (ML), however, the approaches are often rather naïve, using default regression or classification techniques without proper investigation of the suitability of the ML tools. With this work, we bring to the attention of our community the possibility to personalize regression models to specific types of optimization problems. Instead of aiming for a single model that works well across a whole set of possibly diverse problems, our personalized regression approach acknowledges that different models may suite different types of problems. Going one step further, we also investigate the impact of selecting not a single regression model per problem, but personalized ensembles. We test our approach on predicting the performance of numerical optimization heuristics on the BBOB benchmark collection. CCS CONCEPTS• Computing methodologies → Continuous space search; Randomized search; • Theory of computation → Random search heuristics. INTRODUCTIONBefore solving a real-world optimization problem via evolutionary or similar methods, users need to decide which of the many possible algorithms to apply (the algorithm selection (AS) problem) and how to set its parameters (algorithm configuration (AC)).

OPTION: OPTImization Algorithm Benchmarking ONtology

“…RDF Schema (RDFS) 4 is another semantic technology standard that is an extension of the RDF data model and provides essential elements for describing ontologies, such as classes and properties (relations). The Web Ontology Language (OWL) 5 is a collection of representation languages for authoring ontologies with different levels of expressivity. Data stored in a graph format (e.g., as an RDF graph) integrated into an ontology is commonly referred to as a knowledge base.…”

“…ELA provides low-level features computed from a sample of observations for a given problem instance. It has already been shown that representing the characteristics of problem instances using ELA features can provide promising results in automated prediction of algorithm performance [5], [6], automated algorithm selection [7], [8], and automated algorithm configuration [9]. However, calculating the ELA features can be a computationally intensive step.…”

RF+clust for Leave-One-Problem-Out Performance Prediction

“…The work [7] brings to attention the possibility to personalize regression models (Decision Tree, Random Forest and Bagging Tree Regression) to specific types of optimization problems. Instead of aiming for a single model that works well across a whole set of possibly diverse problems, the personalized regression approach acknowledges that different models may suite different problem types.…”

Explainable Landscape-Aware Optimization Performance Prediction

“…It showed that different ML models and different hyper-parameters are recommended to be used for different optimization algorithm prediction. Instead of learning one STR model that performs the best on average across all benchmark problems, Eftimov et al [26] investigated personalized performance regression models for each blackbox optimization problem. For this purpose, ensemble of treebased STR models have been selected for each benchmark optimization problem, which in general decreased the prediction errors within each problem.…”