PG3: Policy-Guided Planning for Generalized Policy Generation

Abstract

Reliable navigation systems have a wide range of applications in robotics and autonomous driving. Current approaches employ an open-loop process that converts sensor inputs directly into actions. However, these open-loop schemes are challenging to handle complex and dynamic real-world scenarios due to their poor generalization. Imitating human navigation, we add a reasoning process to convert actions back to internal latent states, forming a two-stage closed loop of perception, decision-making, and reasoning. Firstly, VAE-Enhanced Demonstration Learning endows the model with the understanding of basic navigation rules. Then, two dual processes in RL-Enhanced Interaction Learning generate reward feedback for each other and collectively enhance obstacle avoidance capability. The reasoning model can substantially promote generalization and robustness, and facilitate the deployment of the algorithm to real-world robots without elaborate transfers. Experiments show our method is more adaptable to novel scenarios compared with state-of-the-art approaches.

Generalized Planning in PDDL Domains with Pretrained Large Language Models

“…Domains. The first six domains (and tasks) are taken directly from the previous work by Yang et al (2022). Of these, four (Gripper, Miconic, Ferry, Spanner) are standard planning benchmarks and the other two (Delivery, Forest) were introduced by that work.…”

“…We evaluate the main approach, four ablations, and four baselines. The baselines are taken from the work by Yang et al (2022); see that work for details.…”

“…• GPT-3.5: GPT-3.5 with CoT summarization and automated debugging. • PG3: The generalized planning approach proposed by Yang et al (2022). The synthesized programs are goalconditioned policies implemented as lifted decision lists.…”

“…In our experiments, we evaluate this approach on seven PDDL domains: six from recent work in generalized planning (Yang et al 2022), and a seventh novel domain. We find that the approach is a strong baseline compared to existing generalized planning approaches.…”

“…One pervasive challenge is that there are often many valid plans for any given task, and only some of these plans are consistent with a simple generalized plan. PG3 addresses this challenge by using candidate generalized plans (represented as lifted decision list goal-conditioned policies) to constrain the generation of example plans (Yang et al 2022). We use PG3 as the main point of comparison in experiments.…”