ASNets: Deep Learning for Generalised Planning

Abstract

In this paper, we discuss the learning of generalised policies for probabilistic and classical planning problems using Action Schema Networks (ASNets). The ASNet is a neural network architecture that exploits the relational structure of (P)PDDL planning problems to learn a common set of weights that can be applied to any problem in a domain. By mimicking the actions chosen by a traditional, non-learning planner on a handful of small problems in a domain, ASNets are able to learn a generalised reactive policy that can quickly solve much larger instances from the domain. This work extends the ASNet architecture to make it more expressive, while still remaining invariant to a range of symmetries that exist in PPDDL problems. We also present a thorough experimental evaluation of ASNets, including a comparison with heuristic search planners on seven probabilistic and deterministic domains, an extended evaluation on over 18,000 Blocksworld instances, and an ablation study. Finally, we show that sparsity-inducing regularisation can produce ASNets that are compact enough for humans to understand, yielding insights into how the structure of ASNets allows them to generalise across a domain.

Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width: Extended Version

“…The use of numerical features that can be incremented and decremented qualitatively is inspired by QNPs (Srivastava et al 2011;Bonet and Geffner 2020). Other works aimed at learning generalized policies or plans include planning programs (Segovia, Jiménez, and Jonsson 2016), logical programs (Silver et al 2020), and deep learning approaches (Groshev et al 2018;Bajpai, Garg, and Mausam 2018;Toyer et al 2020), some of which have been used to learn heuristics (Shen, Trevizan, and Thiébaux 2020;Karia and Srivastava 2021).…”

Sample-efficient Iterative Lower Bound Optimization of Deep Reactive Policies for Planning in Continuous MDPs

“…In the past decade, deep learning (DL) methods have demonstrated remarkable success in a variety of complex applications in computer vision, natural language, and signal processing (Krizhevsky, Sutskever, and Hinton 2017;Hinton et al 2012;Bengio, Lecun, and Hinton 2021). More recently, a variety of work has sought to leverage DL tools for planning and policy learning in a large variety of deterministic and stochastic decision-making domains (Wu, Say, and Sanner 2017;Wu, Say, and Sanner 2020;Say et al 2020;Scaroni et al 2020;Say 2021;Toyer et al 2020;Garg, Bajpai, and Mausam 2020).…”

“…All rights reserved. domain instantiations of these relational models (Groshev et al 2018;Toyer et al 2018;Bajpai, Garg, and Mausam 2018;Mausam 2019, 2020;Toyer et al 2020). Other recent work has investigated planning by discrete and mixed integer optimization in learned discrete neural network models of planning domains (Say and Sanner 2018;Say et al 2020).…”

Learning Generalized Policies Without Supervision Using GNNs

“…In this paper, we consider the problem of learning generalized policies for classical planning domains using graph neural networks (Scarselli et al, 2008;Hamilton, 2020) from small instances represented in lifted STRIPS. The problem has been considered before but using neural architectures that are more complex and with results that are often less crisp, involving in certain cases heuristic information or search (Toyer et al, 2020;Garg et al, 2020;Rivlin et al, 2020;Karia and Srivastava, 2021;Shen et al, 2020). We use a simple and general GNN architecture and aim at obtaining crisp experimental results and a deeper understanding: either the policy greedy in the learned value function achieves close to 100% generalization over instances larger than those used in training, or the failure must be understood and, possibly fixed, using logical methods.…”