Automatic construction of optimal static sequential portfolios for AI planning and beyond

Abstract

In recent years the notion of portfolio has been revived with the aim of improving the performance of modern solvers. For example, Fast Downward Stone Soup and SATzilla have shown an excellent performance at the International Planning and SAT Competitions respectively. However, a deeper understanding of the limits and possibilities of portfolios is still missing. Most approaches to the study of portfolios are purely empirical. Thus, we propose a theoretically-grounded method based on Mixed-Integer Programming named gop. It addresses, among others, three main issues: how to derive an upper bound on the solvers performance for a given set of problem solving tasks; how to analyze the utility of training instances when designing portfolios; and how to configure a high performance portfolio for a given training set which generalizes very well on unseen instances. Experimental results both with data from the International Planning Competitions 2008 and 2011 and the SAT Competition 2013 show that this approach significantly outperforms others under the same conditions. Indeed, MIPSat, the sequential SAT portfolio automatically configured with gop, won the silver medal in the Open track of the SAT Competition 2013. In addition, MIPlan, the planning system which is able to automatically generate a portfolio configuration for a specific planning domain using gop, won the learning track of the International Planning Competition 2014.

What you always wanted to know about the deterministic part of the International Planning Competition (IPC) 2014 (but were too afraid to ask)

“…For information about portfolio approaches in planning, the interested reader is referred to Núñez et al . (2015), Rizzini et al . (2015), Vallati et al .…”

“…In terms of learning approaches, the learning track of IPC 2014 compared the state of the art of learning for planning. For information about portfolio approaches in planning, the interested reader is referred to Núñez et al (2015), Rizzini et al (2015), Vallati et al (2015b); a thorough discussion about mining IPC 2011 results can be found in Cenamor et al (2012). Table 8 shows the results of the performed analysis in terms of selected planners, IPC score and coverage of portfolios of different sizes, on the tracks of IPC 2014.…”

On the Configuration of Robust Static Parallel Portfolios for Efficient Plan Generation

“…Unlike static portfolios, instance-specific portfolios require additional knowledge to be extracted by both training and testing instances, under the form of instance features. MIPlan [14,15] exploits a Mixed-Integer Programming approach for combining planners into static portfolios, either sequential or parallel. Portfolios are optimised to maximise the probability of providing the best available quality plans at any point in time.…”

“…In particular, much work has been done in the area of sequential portfolios, where selected planning engines are executed sequentially on a single CPU. Well-known examples include approaches such as PbP [5], Cedalion [18], and MIPlan [14], which are able to configure static sequential portfolios, and systems like IBaCoP [2], which instead aim at configuring instance-specific portfolios. Static approaches configure portfolios once, and then re-use the same configuration for any testing instance.…”

Automatic construction of parallel portfolios via algorithm configuration

“…The same limitation applies to existing approaches that combine algorithm selection and scheduling, notably CPHydra [60], which also relies on cheaply computable features of the problem instances to be solved and selects multiple solvers to be run in parallel. Two further, conceptually related approaches are aspeed [35] and MIPSAT [59], which compute (parallel) algorithm schedules by taking advantage of the modeling and solving capacities of Answer Set Programming (ASP [10,25]) and Mixed Integer Programming (MIP; [61,64]), respectively.…”