Hallucinative Topological Memory for Zero-Shot Visual Planning

Abstract

In visual planning (VP), an agent learns to plan goal-directed behavior from observations of a dynamical system obtained offline, e.g., images obtained from self-supervised robot interaction. Most previous works on VP approached the problem by planning in a learned latent space, resulting in low-quality visual plans, and difficult training algorithms. Here, instead, we propose a simple VP method that plans directly in image space and displays competitive performance. We build on the semi-parametric topological memory (SPTM) method: image samples are treated as nodes in a graph, the graph connectivity is learned from image sequence data, and planning can be performed using conventional graph search methods. We propose two modifications on SPTM. First, we train an energy-based graph connectivity function using contrastive predictive coding that admits stable training. Second, to allow zero-shot planning in new domains, we learn a conditional VAE model that generates images given a context of the domain, and use these hallucinated samples for building the connectivity graph and planning. We show that this simple approach significantly outperform the SOTA VP methods, in terms of both plan interpretability and success rate when using the plan to guide a trajectory-following controller. Interestingly, our method can pick up nontrivial visual properties of objects, such as their geometry, and account for it in the plans.

Hierarchical Robot Navigation in Novel Environments using Rough 2-D Maps

“…Gao et al [16] also proposes a deep neural network for navigation with a local 2-D map provided as additional signal. Liu et al [17] requires a global context (e.g., an accurate global layout) of the environment to guide the local controller in novel domains. All of the above methods require accurate high-level information; however, they all illustrate promising zero-shot generalization performance in novel tasks, suggesting that high-level maps and contexts are useful for navigation.…”

Visual Goal-Directed Meta-Learning with Contextual Planning Networks

“…Planning through latent space with gradient descent is a core concept shared by several previous works, in which the same embedding is used for both the goal and the current state. The basic idea is that the distance between the latent representation of the current state and the latent representation of the goal could be exploited for planning purposes [10]. The idea can be extended by using a dynamics model to predict the latent state resulting from an action.…”

“…In the limit, zero demonstrations of the new task are provided. This is a special case of zero-shot learning in policy domain, which has become an emerging sub-field within meta-learning [10]. Meta-imitation learning refers to meta-learning approaches that use imitation learning during the meta-training phase.…”

Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation

“…Some are constrained to use local control policies [14,2] or use rule-based algorithms such as beam or A * search [2,15,14] to perform localized path corrections. Others focus on processing long-range observations instead of actions [16] or employ offline pre-training schemes to learn topological structures [12,13,17]. This is challenging since accurate construction of graphs is non-trivial and requires special adaptation to work during real-time navigation [13,17].…”

“…Others focus on processing long-range observations instead of actions [16] or employ offline pre-training schemes to learn topological structures [12,13,17]. This is challenging since accurate construction of graphs is non-trivial and requires special adaptation to work during real-time navigation [13,17]. Under the guidance of natural language instruction, the autonomous agent needs to navigate through the environment from the start state to the target location (red flag).…”