Cognitive Mapping and Planning for Visual Navigation

Gupta, Saurabh; Tolani, Varun; Davidson, James; Levine, Sergey; Sukthankar, Rahul; Malik, Jitendra

doi:10.48550/arxiv.1702.03920

Cited by 44 publications

(71 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Learning methods used for navigation have shown promising results for robot navigation in human domains [3], [16], [17], [18], [19] and in the field [20].…”

Section: B Learning Methods For Robot Navigationmentioning

confidence: 99%

“…Other learning approaches map and plan [19], generate waypoints [17], or use motion primitives [18]. Gupta et al [19] learn a map and how to plan to a goal from egocentric map with end-to-end reinforcement learning. Kumar et al [17] learn to plan at a high level of abstraction (turn left, turn right) from video data.…”

Section: B Learning Methods For Robot Navigationmentioning

confidence: 99%

See 1 more Smart Citation

Passing Through Narrow Gaps with Deep Reinforcement Learning

Tidd¹,

Cosgun²,

Leitner³

et al. 2021

Preprint

View full text Add to dashboard Cite

The DARPA subterranean challenge requires teams of robots to traverse difficult and diverse underground environments. Traversing small gaps is one of the challenging scenarios that robots encounter. Imperfect sensor information makes it difficult for classical navigation methods, where behaviours require significant manual fine tuning. In this paper we present a deep reinforcement learning method for autonomously navigating through small gaps, where contact between the robot and the gap may be required. We first learn a gap behaviour policy to get through small gaps (only centimeters wider than the robot). We then learn a goal-conditioned behaviour selection policy that determines when to activate the gap behaviour policy. We train our policies in simulation and demonstrate their effectiveness with a large tracked robot in simulation and on the real platform. In simulation experiments, our approach achieves 93% success rate when the gap behaviour is activated manually by an operator, and 67% with autonomous activation using the behaviour selection policy. In real robot experiments, our approach achieves a success rate of 73% with manual activation, and 40% with autonomous behaviour selection. While we show the feasibility of our approach in simulation, the difference in performance between simulated and real world scenarios highlight the difficulty of direct sim-to-real transfer for deep reinforcement learning policies. In both the simulated and real world environments alternative methods were unable to traverse the gap.

show abstract

“…Learning methods used for navigation have shown promising results for robot navigation in human domains [3], [16], [17], [18], [19] and in the field [20].…”

Section: B Learning Methods For Robot Navigationmentioning

confidence: 99%

Section: B Learning Methods For Robot Navigationmentioning

confidence: 99%

Passing Through Narrow Gaps with Deep Reinforcement Learning

Tidd¹,

Cosgun²,

Leitner³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A number of learning based approaches has been proposed for visual navigation recently [24; 3; 45; 36; 63; 10; 11; 53]. Modular approaches include CMP [24], DAN [36] and Neural SLAM [10]. However, CMP assumes a known robot location, circumventing the issue of localization.…”

Section: Visual Navigationmentioning

confidence: 99%

Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation

Karkus¹,

Cai²,

Hsu³

2021

Preprint

View full text Add to dashboard Cite

Simultaneous localization and mapping (SLAM) remains challenging for a number of downstream applications, such as visual robot navigation, because of rapid turns, featureless walls, and poor camera quality. We introduce the Differentiable SLAM Network (SLAM-net) along with a navigation architecture to enable planar robot navigation in previously unseen indoor environments. SLAM-net encodes a particle filter based SLAM algorithm in a differentiable computation graph, and learns task-oriented neural network components by backpropagating through the SLAM algorithm. Because it can optimize all model components jointly for the end-objective, SLAM-net learns to be robust in challenging conditions. We run experiments in the Habitat platform with different real-world RGB and RGB-D datasets. SLAM-net significantly outperforms the widely adapted ORB-SLAM in noisy conditions. Our navigation architecture with SLAMnet improves the state-of-the-art for the Habitat Challenge 2020 PointNav task by a large margin (37% to 64% success).

show abstract

“…End-to-End (E2E) Learning for Navigation. There has been a recent interest in employing end-toend learning for training policies for goal-driven navigation [8,9,10,11]. The typical motivation here is to incorporate semantics and common-sense reasoning into navigation policies.…”

Section: Related Workmentioning

confidence: 99%

Combining Optimal Control and Learning for Visual Navigation in Novel Environments

Bansal,

Tolani,

Gupta

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Model-based control is a popular paradigm for robot navigation because it can leverage a known dynamics model to efficiently plan robust robot trajectories. However, it is challenging to use model-based methods in settings where the environment is a priori unknown and can only be observed partially through onboard sensors on the robot. In this work, we address this short-coming by coupling model-based control with learning-based perception. The learning-based perception module produces a series of waypoints that guide the robot to the goal via a collisionfree path. These waypoints are used by a model-based planner to generate a smooth and dynamically feasible trajectory that is executed on the physical system using feedback control. Our experiments in simulated real-world cluttered environments and on an actual ground vehicle demonstrate that the proposed approach can reach goal locations more reliably and efficiently in novel environments as compared to purely geometric mapping-based or end-to-end learning-based alternatives. Our approach does not rely on detailed explicit 3D maps of the environment, works well with low frame rates, and generalizes well from simulation to the real world. Videos describing our approach and experiments are available on the project website 3 .

show abstract

Cognitive Mapping and Planning for Visual Navigation

Cited by 44 publications

References 0 publications

Passing Through Narrow Gaps with Deep Reinforcement Learning

Passing Through Narrow Gaps with Deep Reinforcement Learning

Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation

Combining Optimal Control and Learning for Visual Navigation in Novel Environments

Contact Info

Product

Resources

About