Mixed-Effects Modeling of Optimisation Algorithm Performance

Gagliolo, Matteo; Legrand, Catherine; Birattari, Mauro

doi:10.1007/978-3-642-03751-1_17

Cited by 1 publication

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In statistical terminology, this is an example of longitudinal data, which can be described using mixed ef fects models [10]. In [15] we presented preliminary experiments, showing that nonlinear mixed-effects models can be used to predict the performance of optimization algorithms. The issue with such models is their computational complexity, which scales badly with the size of the sample, rendering their use in time allocation problematic.…”

Section: Discussionmentioning

confidence: 99%

“…To compare fairly with it, we should use the same algorithm set, and preliminarily solve the same set of training instances. 15 The results for both GambleTA and OnG-Exp3 are obtained on the same data, and both algorithms start from scratch, so in this case the comparison is fair. We also report the results of OffG-Oracle [46]: in this case a comparison is obviously not fair, as this allocator is based on prior knowledge of all runtimes.…”

Section: Sat Solvers Competitionsmentioning

confidence: 97%

See 1 more Smart Citation

Algorithm portfolio selection as a bandit problem with unbounded losses

Gagliolo

Schmidhuber

2011

Ann Math Artif Intell

Self Cite

View full text Add to dashboard Cite

We propose a method that learns to allocate computation time to a given set of algorithms, of unknown performance, with the aim of solving a given sequence of problem instances in a minimum time. Analogous meta-learning techniques are typically based on models of algorithm performance, learned during a separate of f line training sequence, which can be prohibitively expensive. We adopt instead an online approach, named GambleTA, in which algorithm performance models are iteratively updated, and used to guide allocation on a sequence of problem instances. GambleTA is a general method for selecting among two or more alternative algorithm portfolios. Each portfolio has its own way of allocating computation time to the available algorithms, possibly based on performance models, in which case its performance is expected to improve over time, as more runtime data becomes available. The resulting exploration-exploitation trade-off is represented as a bandit problem. In our previous work, the algorithms corresponded to the arms of the bandit, and allocations evaluated by the different portfolios were mixed, using a solver for the bandit problem with expert advice, but this required the setting of an arbitrary bound on algorithm runtimes, invalidating the optimal regret of the solver. In this paper, we propose a simpler version of GambleTA, in which the allocators correspond to the arms, such that a single portfolio is selected for each instance. The selection is represented as a bandit problem with partial information, and an unknown bound on losses. We devise a solver for this game, proving a bound on its expected regret. We present experiments based on results from several M. Gagliolo (B) CoMo, Vrije Universiteit Brussel, M. Gagliolo, J. Schmidhuber solver competitions, in various domains, comparing GambleTA with another online method.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Sat Solvers Competitionsmentioning

confidence: 97%