Learning Domain-Independent Planning Heuristics with Hypergraph Networks

Abstract

We present the first approach capable of learning domain-independent planning heuristics entirely from scratch. The heuristics we learn map the hypergraph representation of the delete-relaxation of the planning problem at hand, to a cost estimate that approximates that of the least-cost path from the current state to the goal through the hypergraph. We generalise Graph Networks to obtain a new framework for learning over hypergraphs, which we specialise to learn planning heuristics by training over state/value pairs obtained from optimal cost plans. Our experiments show that the resulting architecture, STRIPS-HGNs, is capable of learning heuristics that are competitive with existing delete-relaxation heuristics including LM-cut. We show that the heuristics we learn are able to generalise across different problems and domains, including to domains that were not seen during training.

Meta Reinforcement Learning for Heuristic Planing

“…There has also been an increasing interest in developing deep learning techniques to improve the performance of automated planning (Fern, Khardon, and Tadepalli 2011). For instance, learning policies (Garg, Bajpai, and Mausam 2020;Groshev et al 2018;Toyer et al 2018;Issakkimuthua, Fern, and Tadepalli 2018;Mausam 2019, 2020;Shen et al 2019), planner selection (Sievers et al 2019;Ma et al 2020;Katz et al 2018) and heuristics (Arfaee, Zilles, and Holte 2011;Groshev et al 2018;Samadi, Felner, and Schaeffer 2008;Thayer, Dionne, and Ruml 2011;Garrett, Kaelbling, and Lozano-Pérez 2016;Shen, Trevizan, and Thiébaux 2020) have been widely explored. Our work fits within the heuristic learning category.…”

“…Recent methods for learning heuristics combine or improve on existing heuristics (Arfaee, Zilles, and Holte 2011;Groshev et al 2018;Samadi, Felner, and Schaeffer 2008;Thayer, Dionne, and Ruml 2011;Garrett, Kaelbling, and Lozano-Pérez 2016;Shen, Trevizan, and Thiébaux 2020). All of these methods use supervised learning but differ in the encoding of the states, proposing, for instance, the use of images (Groshev et al 2018;Ma et al 2020;Katz et al 2018) or sophisticated network models (Shen, Trevizan, and Thiébaux 2020;Toyer et al 2018).…”

“…Recent methods for learning heuristics combine or improve on existing heuristics (Arfaee, Zilles, and Holte 2011;Groshev et al 2018;Samadi, Felner, and Schaeffer 2008;Thayer, Dionne, and Ruml 2011;Garrett, Kaelbling, and Lozano-Pérez 2016;Shen, Trevizan, and Thiébaux 2020). All of these methods use supervised learning but differ in the encoding of the states, proposing, for instance, the use of images (Groshev et al 2018;Ma et al 2020;Katz et al 2018) or sophisticated network models (Shen, Trevizan, and Thiébaux 2020;Toyer et al 2018). A common approach is to do regression on the heuristic values obtained from precomputed plans (Shen, Trevizan, and Thiébaux 2020;Toyer et al 2018;Garrett, Kaelbling, and Lozano-Pérez 2016;Yoon, Fern, and Givan 2008), and for this reason, it is the baseline we used to compare against supervised methods.…”

“…Sokoban Sokoban is a game from IPC-2008, also used to evaluate previous Deep Learning approaches (Shen, Trevizan, and Thiébaux 2020;Groshev et al 2018). The agent must push objects around a grid with the goal of moving them to specific locations.…”

“…All rights reserved. et al Toyer et al 2018;Shen, Trevizan, and Thiébaux 2020). These approaches are based on supervised learning, and learn from the optimal plans of previously solved planning problems.…”