BADGR: An Autonomous Self-Supervised Learning-Based Navigation System

Kahn, Gregory; Abbeel, Pieter; Levine, Sergey

doi:10.1109/lra.2021.3057023

Cited by 190 publications

(148 citation statements)

References 24 publications

Supporting

Mentioning

129

Contrasting

Order By: Relevance

“…Already in 2005, the authors of [15] proposed a system that uses a monocular camera to estimate the distance to obstacles geometrically and use that information to train a policy in simulation. Other works feed RGB images or 2D LiDAR data through a neural network to produce a control command for navigation by learning from expert trajectories [16], [17], [18], [9], via reinforcement learning [19], [20], or fully selfsupervised [21]. In this work, we do not assume the presence of an expert and use depth images.…”

Section: B Related Workmentioning

confidence: 99%

Learning a State Representation and Navigation in Cluttered and Dynamic Environments

Hoeller

Wellhausen

Farshidian

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

In this work, we present a learning-based pipeline to realise local navigation with a quadrupedal robot in cluttered environments with static and dynamic obstacles. Given high-level navigation commands, the robot is able to safely locomote to a target location based on frames from a depth camera without any explicit mapping of the environment. First, the sequence of images and the current trajectory of the camera are fused to form a model of the world using state representation learning. The output of this lightweight module is then directly fed into a target-reaching and obstacle-avoiding policy trained with reinforcement learning. We show that decoupling the pipeline into these components results in a sample efficient policy learning stage that can be fully trained in simulation in just a dozen minutes. The key part is the state representation, which is trained to not only estimate the hidden state of the world in an unsupervised fashion, but also helps bridging the reality gap, enabling successful sim-to-real transfer. In our experiments with the quadrupedal robot ANYmal in simulation and in reality, we show that our system can handle noisy depth images, avoid dynamic obstacles unseen during training, and is endowed with local spatial awareness.

show abstract

Section: B Related Workmentioning

confidence: 99%

Learning a State Representation and Navigation in Cluttered and Dynamic Environments

Hoeller

Wellhausen

Farshidian

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Research in this last category can be further distinguished based on the policy design and supervision used for training. Given the infeasibility of simulation, challenging terrain, lack of a reliable unsupervised self-supervision signal (as used in BADGR [35]), and difficulty of large-scale field experiments, renders reinforcement learning, imitation learning, and self-supervision based methods infeasible for our task [54,47,47,24,35,49,30]. Also, lack of large-scale datasets for training has prevented the use of machine learning (overcanopy datasets e.g.…”

Section: Related Workmentioning

confidence: 99%

Learned Visual Navigation for Under-Canopy Agricultural Robots

Sivakumar

Modi

Gasparino

et al. 2021

Robotics: Science and Systems XVII

View full text Add to dashboard Cite

This paper describes a system for visually guided autonomous navigation of under-canopy farm robots. Low-cost under-canopy robots can drive between crop rows under the plant canopy and accomplish tasks that are infeasible for overthe-canopy drones or larger agricultural equipment. However, autonomously navigating them under the canopy presents a number of challenges: unreliable GPS and LiDAR, high cost of sensing, challenging farm terrain, clutter due to leaves and weeds, and large variability in appearance over the season and across crop types. We address these challenges by building a modular system that leverages machine learning for robust and generalizable perception from monocular RGB images from low-cost cameras, and model predictive control for accurate control in challenging terrain. Our system, CropFollow, is able to autonomously drive 485 meters per intervention on average, outperforming a state-of-the-art LiDAR based system (286 meters per intervention) in extensive field testing spanning over 25 km. I. INTRODUCTIONThis paper describes the design of a visually-guided navigation system for compact, low-cost, under-canopy agricultural robots for commodity row-crops (corn, soybean, sugarcane etc), such as that shown in Figure 1. Our system, called CropFollow, uses monocular RGB images from an on-board front-facing camera to steer the robot to autonomously traverse in between crop rows in harsh, visually cluttered, uneven, and variable real-world agricultural fields. Robust and reliable autonomous navigation of such under-canopy robots has the potential to enable a number of practical and scientific applications: High-throughput plant phenotyping [43,37,68,66,58,25], ultra-precise pesticide treatments, mechanical weeding [41], plant manipulation [17,61], and cover crop planting [64,62] Such applications are not possible with overcanopy larger tractors and UAVs, and are crucial for increasing agricultural sustainability [55,22].Autonomous row-following is a foundational capability for robots that need to navigate between crop rows in agricultural fields. Such robots cannot rely on RTK (Real-Time Kinematic)-GPS [21] based methods which are used for overthe-canopy autonomy (e.g. for drones, tractors, and combine Project website with data and videos: https://ansivakumar.github.io/ learned-visual-navigation/. Correspondence to {av7,girishc}@illinois.edu. * Girish Chowdhary and Saurabh Gupta contributed equally and are listed alphabetically.

show abstract

“…Because large annotated datasets are needed to train such deep networks, the alternative of self-supervised learning approaches has also been employed. In Reference [ 13 ], BADGR—Berkeley Autonomous Driving Ground Robot, an End2End self-supervised learning system, was created to navigate in real-world situations with geometrically distracting obstacles (such as tall grass). It can also take into account terrain preferences, generalize to new environments, and improve on its own by collecting more data.…”

Section: Related Workmentioning

confidence: 99%

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

Trasnea

Ginerica

Zaha

et al. 2021

Sensors

View full text Add to dashboard Cite

Autonomous mobile robots are usually faced with challenging situations when driving in complex environments. Namely, they have to recognize the static and dynamic obstacles, plan the driving path and execute their motion. For addressing the issue of perception and path planning, in this paper, we introduce OctoPath, which is an encoder-decoder deep neural network, trained in a self-supervised manner to predict the local optimal trajectory for the ego-vehicle. Using the discretization provided by a 3D octree environment model, our approach reformulates trajectory prediction as a classification problem with a configurable resolution. During training, OctoPath minimizes the error between the predicted and the manually driven trajectories in a given training dataset. This allows us to avoid the pitfall of regression-based trajectory estimation, in which there is an infinite state space for the output trajectory points. Environment sensing is performed using a 40-channel mechanical LiDAR sensor, fused with an inertial measurement unit and wheels odometry for state estimation. The experiments are performed both in simulation and real-life, using our own developed GridSim simulator and RovisLab’s Autonomous Mobile Test Unit platform. We evaluate the predictions of OctoPath in different driving scenarios, both indoor and outdoor, while benchmarking our system against a baseline hybrid A-Star algorithm and a regression-based supervised learning method, as well as against a CNN learning-based optimal path planning method.

show abstract

BADGR: An Autonomous Self-Supervised Learning-Based Navigation System

Cited by 190 publications

References 24 publications

Learning a State Representation and Navigation in Cluttered and Dynamic Environments

Learning a State Representation and Navigation in Cluttered and Dynamic Environments

Learned Visual Navigation for Under-Canopy Agricultural Robots

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

Contact Info

Product

Resources

About