Statistical Selection Among Problem-Solving Methods.

Abstract

The choice of the right problem-solving method, from available methods, is a crucial skill for experts in many areas. We describe a technique for automatic selection among methods, based on analysis of their past performances. We formalize the statistical problem involved in choosing an e cient method, derive a solution to this problem, and describe a selection algorithm. The algorithm not only chooses among available methods, but also decides when to abandon the chosen method, if it takes too much time. We then extend the basic statistical technique to account for problem sizes and similarity among problems. We apply this technique to select among search engines in the prodigy system, and then test the selection technique on arti cially generated performance data, with several probability distributions.

Automatic Algorithm Selection for Complex Simulation Problems

“…2.2), this selection process is called theoretical. It subsumes rigorous mathematical deductions of the best algorithm for some problem features and user criteria, but also intuitive decisions, e.g., as mentioned in [81,257], as long as these are not based on any experimental data that has been recorded beforehand. Intuitive algorithm selection often implicitly steers algorithm design, e.g., when a developer decides to implement a specific synchronization scheme for a parallel and distributed simulation engine.…”

“…Rice already proposed algorithm selection for operation system task schedulers [272]. It is also relevant for adaptive middleware [22,230], planning [81], hardware design [262], and generally all application domains in which:…”

Algorithm Selection: From Meta-Learning to Hyper-Heuristics

“…The current state of the art is such that often there are many algorithms and systems for solving the same k i n d o f p r o b l e m ; e a c h w i t h i t s o w n performance on a particular problem. Machine learning is an established method of addressing ASP (Lobjois & Lemâitre, 1998;Fink, 1998). Given the performance of each algorithm on a set of training problems, we try to predict the performance on unseen problems (Kotthoff et al, 2011).…”

“…Since the right decisions may depend on the problem size and parameters, the machine characteristics and load, the data distribution, and other uncertain factors, this can be quite challenging. Some works treats algorithms in a black-box manner: each time a single algorithm is selected and applied to the given instance then a regression analysis or machine learning techniques are used to build a predictive model of the performance of the algorithms given the features of the instances (Lobjois & Lemâitre, 1998;Fink, 1998;Leyton-Brown et al, 2003;Ali & Smith, 2006). Other works focus on dynamic selection of algorithm components while the instance is being solved.…”

Dominion: An Architecture-Driven Approach to Generating Efficient Constraint Solvers

“…The relevant template specifications are not shown due to lack of space. The reader is referred to existing literature on algorithm selection [10], [11] and algorithm portfolios [12] for more details on possible techniques for the analyser.…”