Algorithm portfolio selection as a bandit problem with unbounded losses

Abstract

We propose a method that learns to allocate computation time to a given set of algorithms, of unknown performance, with the aim of solving a given sequence of problem instances in a minimum time. Analogous meta-learning techniques are typically based on models of algorithm performance, learned during a separate of f line training sequence, which can be prohibitively expensive. We adopt instead an online approach, named GambleTA, in which algorithm performance models are iteratively updated, and used to guide allocation on a sequence of problem instances. GambleTA is a general method for selecting among two or more alternative algorithm portfolios. Each portfolio has its own way of allocating computation time to the available algorithms, possibly based on performance models, in which case its performance is expected to improve over time, as more runtime data becomes available. The resulting exploration-exploitation trade-off is represented as a bandit problem. In our previous work, the algorithms corresponded to the arms of the bandit, and allocations evaluated by the different portfolios were mixed, using a solver for the bandit problem with expert advice, but this required the setting of an arbitrary bound on algorithm runtimes, invalidating the optimal regret of the solver. In this paper, we propose a simpler version of GambleTA, in which the allocators correspond to the arms, such that a single portfolio is selected for each instance. The selection is represented as a bandit problem with partial information, and an unknown bound on losses. We devise a solver for this game, proving a bound on its expected regret. We present experiments based on results from several M. Gagliolo (B) CoMo, Vrije Universiteit Brussel, M. Gagliolo, J. Schmidhuber solver competitions, in various domains, comparing GambleTA with another online method.

Algorithm Selector and Prescheduler in the ICON Challenge

“…The performance model is usually built by applying machine learning approaches onto a dataset reporting the algorithm performances on a comprehensive set of benchmark problem instances (with the exception of [5], using a multi-armed bandit approach). Such machine learning approaches range from k-nearest neighbors [16] to ridge regression [23], random forests [24], collaborative filtering [20,15], or learning to rank approaches [17].…”

Static, Dynamic, and Adaptive Heterogeneity in Distributed Smart Camera Networks

“…Furthermore, as cameras learn, the learning problem facing the other cameras changes in response. We therefore consider that a camera is faced with a non-stationary online algorithm selection problem [Gagliolo and Schmidhuber 2011]. Our approach is to consider this as an variant of the multi-armed bandit problem [Auer et al 2002].…”

Bandits With Heavy Tail

“…A prominent example is the distribution of delays in end-to-end network routing [5], a typical application domain for bandits-see, e.g., [2]. Another interesting example is the distribution of running times of heuristics for solving hard combinatorial problems [6], where bandit algorithms have been used to select the heuristics [7].…”