Bounded Suboptimal Search with Learned Heuristics for Multi-Agent Systems

Abstract

A wide range of discrete planning problems can be solved optimally using graph search algorithms. However, optimal search quickly becomes infeasible with increased complexity of a problem. In such a case, heuristics that guide the planning process towards the goal state can increase performance considerably. Unfortunately, heuristics are often unavailable or need manual and time-consuming engineering. Building upon recent results on applying deep learning to learn generalized reactive policies, we propose to learn heuristics by imitation learning. After learning heuristics based on optimal examples, they are used to guide a classical search algorithm to solve unseen tasks. However, directly applying learned heuristics in search algorithms such as A∗ breaks optimality guarantees, since learned heuristics are not necessarily admissible. Therefore, we (i) propose a novel method that utilizes learned heuristics to guide Focal Search A∗, a variant of A∗ with guarantees on bounded suboptimality; (ii) compare the complexity and performance of jointly learning individual policies for multiple robots with an approach that learns one policy for all robots; (iii) thoroughly examine how learned policies generalize to previously unseen environments and demonstrate considerably improved performance in a simulated complex dynamic coverage problem.

Geometric Task Networks: Learning Efficient and Explainable Skill Coordination for Object Manipulation

“…An important technique used in this work is to learn an imitation policy from an expert solver or human, such that this policy can be used online but with less solution time or improved generalization. Imitation learning has been widely used for various purposes, e.g., autonomous driving, robot motion control [12], [25], and multi-robot coordination [8], [26]. Most of the these work has a strong focus on learning low-level control policy from raw visual inputs, without considering high-level tasks.…”

“…Most of the these work has a strong focus on learning low-level control policy from raw visual inputs, without considering high-level tasks. Furthermore, training data can be generated from a complete solver [8], [25], [27] or expert demonstrations [10]. They are commonly represented by deep neural networks (DNN) such as CNN [8], [26], GNN [25], and VAE [28].…”

“…Furthermore, training data can be generated from a complete solver [8], [25], [27] or expert demonstrations [10]. They are commonly represented by deep neural networks (DNN) such as CNN [8], [26], GNN [25], and VAE [28]. High-dimensional sensory inputs such as images in [12], [26] or point clouds in [20] enable direct reasoning over raw inputs, while direct state information such as object poses in [25] provides easy interfaces to the existing motion planners.…”

“…Due to the repetitive nature of industrial processes, an intriguing question to ask is "Can we solve the problem faster after solving it 100 times?". Thus some recent work proposes to learn such heuristics from expert solutions or via reinforcement learning, see e.g., [8], [9], [10]. However, most of these approaches require enormous amount of training data even for relatively simple tasks.…”

Value Iteration Networks on Multiple Levels of Abstraction

“…by using AVIN to generate an informed heuristic for A * . Different methods to combine search-and learning-based planners have been proposed in [6], [13] and [23].…”