SNNAP: Solver-Based Nearest Neighbor for Algorithm Portfolios

Abstract

A Mamma e Papà e ai miei fratelli AbstractThe success of portfolio algorithms in competitions in the area of combinatorial problem solving, as well as in practice, has motivated interest in the development of new approaches to determine the best solver for the problem at hand. Yet, although there are a number of ways in which this decision can be made, it always relies on a rich set of features to identify and distinguish the structure of the problem instances.In this thesis, however, it is firstly shown how not all the features in the problem have the same relevancy. Then it is presented how one of the most successful portfolio approaches, ISAC, can be augmented by taking into account the past performance of solvers as part of the feature vector. Testing on a variety of SAT datasets, it is here proved how the new formulation continuously outperforms an unmodified/standard version of ISAC.This thesis presents a novel strategy that tackles the problem of algorithm portfolios applying machine learning techniques and improves a state-of-the-art portfolio algorithms approaches (ISAC).Starting from the basic assumptions behind the standard ISAC methodology that the given collection of features can be used to correctly identify the structure of each instance, an important step has been to be able to improve clustering, grouping together instances that prefer to be solved by the same solver. It has been created, and here is presented, a methodology to redefine the feature vector including past performance of the solvers. The final result is named SNNAP: Solver-based Nearest Neighbor for Algorithm Portfolios which is here presented.

An enhanced features extractor for a portfolio of constraint solvers

“…Numerical results on both SAT and CSP domains show that the number of features can be significantly reduced while often providing considerable performances gains. Moreover, in [8] the authors introduce SNNAP (Solver-based Nearest Neighbors for Algorithm Portfolios), an alternative view of ISAC which uses the existing features to predict the best three solvers for a particular instance. A brand new classifier that selects solvers based on a Cost-Sensitive Hierarchical Clustering (CSHC) model is presented in [23].…”

“…Unfortunately, we have not seen significant improvements: all the performance gains were always below 1%. 8 However, it is worth noting that merely removing constant features and scaling them in a given range could lead to major enhancements with a small computational effort. Consider for instance Figure 3 that shows a comparison between the performances obtained by 3S on Benchmark B, by using normalized and not normalized features.…”

“…As it can be seen, the difference is considerable: the performance gap that can be obtained by normalizing the features ranges from a minimum of 3.08% to a maximum of 4.68%. 8 We also tried to apply the filtering techniques of [20] exploiting ISAC code; unfortunately, this proved to be too time consuming (filtering of a single fold took more than a day of computation).…”

sunny-as2: Enhancing SUNNY for Algorithm Selection

“…Another approach based on k-NN is SNNAP (Collautti, Malitsky, Mehta, & O'Sullivan, 2013), which first predicts the performance of each algorithm with regression models and then uses this information for a k-NN approach in the predicted performance space. As for 3S, SNNAP was inspired by the ISAC approach.…”

claspfolio2: Advances in Algorithm Selection for Answer Set Programming

“…Especially on SAT problems, automatic algorithm selectors have achieved impressive performance improvements in the last decade. SATzilla (Xu et al 2008;2007;2012) predicted algorithm performance by means of ridge regression until 2009 and nowadays uses a pairwise voting scheme based on random forests; ISAC (Kadioglu et al 2010) clusters instances in the instance feature space and uses a nearest neighbour approach on cluster centers for algorithm selection; 3S (Kadioglu et al 2011;Malitsky et al 2013) uses k-NN in the feature space and introduces pre-solving schedules computed by Integer Linear Programming and cost-sensitive clustering; SNAPP (Collautti et al 2013) predicts algorithm performance based on instance features and chooses an algorithm based on the similarity of the predicted performances. All these systems are specialized on a single approach.…”