The first learning track of the international planning competition

Abstract

The International Planning Competition is a biennial event organized in the context of the International Conference on Automated Planning and Scheduling. The 2008 competition included, for the first time, a learning track for comparing approaches for improving automated planners via learning. In this paper, we describe the structure of the learning track, the planning domains used for evaluation, the participating systems, the results, and our observations. Towards supporting the goal of domain-independent learning, one of the key features of the competition was to disallow any code changes or parameter tweaks after the training domains were revealed to the participants. The competition results show that at this stage no learning for planning system outperforms state-of-the-art planners in a domain independent manner across a wide range of domains. However, they appear to be close to providing such performance. Evaluating learning for planning systems in a blind competition raises important questions concerning criteria that should be taken into account in future competitions.

Exploiting Macro-actions and Predicting Plan Length in Planning as Satisfiability

“…Table 3 gives the percentage of solved problems, the average CPU time and the speed score of the original version of SatPlan and the proposed version using learned planning horizons (abbreviated with SatPlan+H). The speed score was first introduced and used by the organizers of the 6th International Planning Competition [6] for evaluating the relative performance of the competing planners, and since then it has become a standard method for comparing planning systems. The speed score of a system s is defined as the sum of the speed scores assigned to s over all the considered problems.…”

“…Learning for planning is an important research field in automated planning research, that, as demonstrated by the last two planning competitions [6,3], in the recent years has received considerable attention in the planning community. Starting from the PDDL formalization of a planning problem, the current learning techniques for deterministic (classical) planning aim at automatically generating additional knowledge about the problem, and at effectively using it to improve the performance of a planner.…”

A Guide to Portfolio-Based Planning

“…The only existing system that is able to configure a domainspecific portfolio (also considering additional domain-related knowledge) for minimizing runtime is PbP (and its latest version, PbP2). Analyzing the runtimesconfigured portfolios that PbP generated for IPC6-7 benchmark domains [3,2], it is easy to note that usually a single planner (possibly with additional knowledge extracted from the domain under the form of macro-actions) is selected. It would be interesting, for all the Automated Planning community, to offer an in-depth analysis for better understanding this behaviour.…”

“…Very recently, a number of planners based on portfolio approach have been developed, and achieved impressive results in the last editions of the International Planning Competition (IPC6-7) [3,2]: they won, or got very close to, in every track they took part. These include the deterministic track, learning track and multicore track.…”

Portfolio-based planning: State of the art, common practice and open challenges

“…Recently, a number of planners based on the portfolio approach have been developed, and achieved impressive results in the last editions of the International Planning Competition (IPC6-8) [11,9,34]: they won, or got very close to winning, in almost every track in which they took part. Achieved results, and the aforementioned observations, let us presume that the future of AI planning will not only be focused on developing new planning algorithms, as in the last decades, but also on designing promising techniques for combining and exploiting them.…”