Making Choices Using Structure at the Instance Level within a Case Based Reasoning Framework

Gebruers, Cormac; Guerri, Alessio; Hnich, Brahim; Milano, Michela

doi:10.1007/978-3-540-24664-0_27

Cited by 19 publications

(13 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Haim & Walsh [37] extended linear methods to the problem of making online estimates of SAT solver runtimes. Several researchers have applied supervised classification to select the fastest algorithm for a problem instance [33,29,34,30,120] or to judge whether a particular run of a randomized algorithm would be good or bad [43] (in contrast to our topic of predicting performance directly using a regression model). In the machine learning community, meta-learning aims to predict the accuracy of learning algorithms [111].…”

Section: Related Work On Predicting Runtime Of Parameterless Algorithmsmentioning

confidence: 99%

Algorithm runtime prediction: Methods & evaluation

Hutter

Hoos

et al. 2014

Artificial Intelligence

357

328

View full text Add to dashboard Cite

Perhaps surprisingly, it is possible to predict how long an algorithm will take to run on a previously unseen input, using machine learning techniques to build a model of the algorithm's runtime as a function of problem-specific instance features. Such models have important applications to algorithm analysis, portfolio-based algorithm selection, and the automatic configuration of parameterized algorithms. Over the past decade, a wide variety of techniques have been studied for building such models. Here, we describe extensions and improvements of existing models, new families of models, andperhaps most importantly-a much more thorough treatment of algorithm parameters as model inputs. We also comprehensively describe new and existing features for predicting algorithm runtime for propositional satisfiability (SAT), travelling salesperson (TSP) and mixed integer programming (MIP) problems. We evaluate these innovations through the largest empirical analysis of its kind, comparing to a wide range of runtime modelling techniques from the literature. Our experiments consider 11 algorithms and 35 instance distributions; they also span a very wide range of SAT, MIP, and TSP instances, with the least structured having been generated uniformly at random and the most structured having emerged from real industrial applications. Overall, we demonstrate that our new models yield substantially better runtime predictions than previous approaches in terms of their generalization to new problem instances, to new algorithms from a parameterized space, and to both simultaneously. 102 21 59 109 6E3 1E3 1E3 6E3 Minisat 2.0-HAND 1903 3600 1.25 0.57 0.53 0.52 0.51 74 14 48 79 3E3 96 179 3E3 Minisat 2.0-RAND 2497 3600 0.82 0.39 0.38 0.37 0.37 59 23 52 65 221 35 145 230 Minisat 2.0-INDU 1146 3600 0.94 0.58 0.57 0.55 0.52 222 24 85 231 6E3 1E3 1E3 6E3 Minisat 2.0-SWV-IBM 466 3600 0.85 0.16 0.17 0.17 0.17 98 8.4 59 102 1E3 74 153 1E3 Minisat 2.0-IBM 834 3600 1.1 0.21 0.25 0.21 0.19 130 11 78 136 1E3 74 153 1E3 Minisat 2.0-SWV 0.89 5.32 0.25 0.08 0.09 0.08 0.08 57 4.9 34 59 217 17 123 226 CryptoMinisat-INDU 1921 3600 1.1 0.81 0.73 0.74 0.72 222 24 85 231 6E3 1E3 1E3 6E3 CryptoMinisat-SWV-IBM 873 3600 1.07 0.47 0.5 0.49 0.48 98 8.4 59 102 1081 74 153 1103 CryptoMinisat-IBM 1178 3600 1.2 0.42 0.45 0.42 0.41 130 11 78 136 1081 74 153 1103 CryptoMinisat-SWV 486 3600 0.89 0.51 0.53 0.49 0.51 57 4.9 34 59 217 17 123 226 SPEAR-INDU 1685 3600 1.01 0.67 0.62 0.61 0.58 222 24 85 231 6E3 1E3 1E3 6E3 SPEAR-SWV-IBM 587 3600 0.97 0.38 0.39 0.39 0.38 98 8.4 59 102 1E3 74 153 1E3 SPEAR-IBM 1004 3600 1.18 0.39 0.42 0.42 0.38 130 11 78 136 1E3 74 153 1E3 SPEAR-SWV 60 3600 0.54 0.36 0.34 0.34 0.34 57 4.9 34 59 217 17 123 226 tnm-RANDSAT 568 3600 1.05 0.88 0.97 0.9 0.88 63 26 56 70 221 35 145 230 SAPS-RANDSAT 1019 3600 1 0.67 0.71 0.65 0.66 63 26 56 70 221 35 145 230 CPLEX-BIGMIX 719 3600 0.96 0.84 0.85 0.63 0.64 17 0.13 6.7 23 1E4 6.6 54 1E4 Gurobi-BIGMIX 992 3600 1.31 1.28 1.31 1.19 1.17 17 0.13 6.7 23 1E4 6.6 54 1E4 SCIP-BIGMIX 1153 3600 0.77 0.67 0.72 0.58...

show abstract

Section: Related Work On Predicting Runtime Of Parameterless Algorithmsmentioning

confidence: 99%

Algorithm runtime prediction: Methods & evaluation

Hutter

Hoos

et al. 2014

Artificial Intelligence

357

328

View full text Add to dashboard Cite

show abstract

“…It also employs a portfolio-based design [25]. Classification has been used in many works, e.g., [18,21]. Unlike our work, these approaches use only unweighted classification problems.…”

Section: Related Workmentioning

confidence: 98%

MUX: algorithm selection for software model checkers

Tulsian

Kanade

Kumar

et al. 2014

Proceedings of the 11th Working Conference on Mining Software Repositories

View full text Add to dashboard Cite

With the growing complexity of modern day software, software model checking has become a critical technology for ensuring correctness of software. As is true with any promising technology, there are a number of tools for software model checking. However, their respective performance trade-offs are difficult to characterize accurately -making it difficult for practitioners to select a suitable tool for the task at hand. This paper proposes a technique called MUX that addresses the problem of selecting the most suitable software model checker for a given input instance. MUX performs machine learning on a repository of software verification instances. The algorithm selector, synthesized through machine learning, uses structural features from an input instance, comprising a program-property pair, at runtime and determines which tool to use.We have implemented MUX for Windows device drivers and evaluated it on a number of drivers and model checkers. Our results are promising in that the algorithm selector not only avoids a significant number of timeouts but also improves the total runtime by a large margin, compared to any individual model checker. It also outperforms a portfolio-based algorithm selector being used in Microsoft at present. Besides, MUX identifies structural features of programs that are key factors in determining performance of model checkers.

show abstract

“…However, avoiding the training phase can be clearly advantageous in terms of simplicity and flexibility: new information can be used to improve the predictions without rebuilding the prediction model. For this reasons some lazy approaches have been proposed in the literature (e.g., see [35,37,34,11,43,4]). A further distinction can be made between algorithms that run just one solver and those that schedule more solvers.…”

Section: B Algorithm Selectionmentioning

confidence: 99%

An Extensive Evaluation of Portfolio Approaches for Constraint Satisfaction Problems

Amadini¹,

Gabbrielli²,

Mauro³

2016

IJIMAI

View full text Add to dashboard Cite

-In the context of Constraint Programming, a portfolio approach exploits the complementary strengths of a portfolio of different constraint solvers. The goal is to predict and run the best solver(s) of the portfolio for solving a new, unseen problem. In this work we reproduce, simulate, and evaluate the performance of different portfolio approaches on extensive benchmarks of Constraint Satisfaction Problems. Empirical results clearly show the benefits of portfolio solvers in terms of both solved instances and solving time.

show abstract

Making Choices Using Structure at the Instance Level within a Case Based Reasoning Framework

Cited by 19 publications

References 3 publications

Algorithm runtime prediction: Methods & evaluation

Algorithm runtime prediction: Methods & evaluation

MUX: algorithm selection for software model checkers

An Extensive Evaluation of Portfolio Approaches for Constraint Satisfaction Problems

Contact Info

Product

Resources

About