CNN-based single image obstacle avoidance on a quadrotor

Chakravarty, Punarjay; Kelchtermans, Klaas; Roussel, Tom; Wellens, Stijn; Tuytelaars, Tinne; Eycken, Luc Van

doi:10.1109/icra.2017.7989752

Cited by 64 publications

(30 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This learning ability benefits the autonomous agent to maneuver safely in environments without prior knowledge of the surroundings, as well as in environments with moving obstacles. Furthermore, the agent is competent to move deftly near corners (refer supplementary video) which has been found to be a challenging task for the previously proposed controllers ( [19], [23]). • The reward function is designed by considering the energy constraints on aerial systems and time factor in navigation tasks.…”

Section: Discussionmentioning

confidence: 99%

“…The proposed conditional GAN is initially trained on a total of 90, 000 RGB-D image pairs collected from the Gazebo [39] simulated environments each having different characteristics. We have a total of 22 different simulated indoor environments, of which few are inspired from [19] while the rest are self designed. The environments consist of broad and narrow hallways, small and large enclosed areas with floorings ranging from asphalt to artificial turf.…”

Section: A Depth Network Settingsmentioning

confidence: 99%

“…Varied obstacle avoidance techniques in conjunction with depth prediction are proposed in [19]- [23]. [19] proposes a behavior arbitration scheme to obtain the yaw and pitch angles for the UAV to avoid an obstacle and for navigation in general. Trajectory planning using obstacle bounding boxes and depth estimation is explored in [20].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge

Singla

Padakandla

Bhatnagar

2021

IEEE Trans. Intell. Transport. Syst.

156

View full text Add to dashboard Cite

This paper presents our method for enabling a UAV quadrotor, equipped with a monocular camera, to autonomously avoid collisions with obstacles in unstructured and unknown indoor environments. When compared to obstacle avoidance in ground vehicular robots, UAV navigation brings in additional challenges because the UAV motion is no more constrained to a well-defined indoor ground or street environment. Horizontal structures in indoor and outdoor environments like decorative items, furnishings, ceiling fans, sign-boards, tree branches etc., also become relevant obstacles unlike those for ground vehicular robots. Thus, methods of obstacle avoidance developed for ground robots are clearly inadequate for UAV navigation. Current control methods using monocular images for UAV obstacle avoidance are heavily dependent on environment information. These controllers do not fully retain and utilize the extensively available information about the ambient environment for decision making. We propose a deep reinforcement learning based method for UAV obstacle avoidance (OA) and autonomous exploration which is capable of doing exactly the same. The crucial idea in our method is the concept of partial observability and how UAVs can retain relevant information about the environment structure to make better future navigation decisions. Our OA technique uses recurrent neural networks with temporal attention and provides better results compared to prior works in terms of distance covered during navigation without collisions. In addition, our technique has a high inference rate (a key factor in robotic applications) and is energy-efficient as it minimizes oscillatory motion of UAV and reduces power wastage.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: A Depth Network Settingsmentioning

confidence: 99%

See 1 more Smart Citation

Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge

Singla

Padakandla

Bhatnagar

2021

IEEE Trans. Intell. Transport. Syst.

156

View full text Add to dashboard Cite

show abstract

“…In the supervised case [8], [13], thousands of image-depth map tuples collected from a depth sensor (like the Microsoft Kinect) are used to train a Convolutional Neural Network (CNN) to determine depth from a single image. However, this requires the use of depth cameras, which is not always available.…”

Section: B Single Image Depthmentioning

confidence: 99%

GEN-SLAM: Generative Modeling for Monocular Simultaneous Localization and Mapping

Chakravarty

Narayanan

Roussel

2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

We present a Deep Learning based system for the twin tasks of localization and obstacle avoidance essential to any mobile robot. Our system learns from conventional geometric SLAM, and outputs, using a single camera, the topological pose of the camera in an environment, and the depth map of obstacles around it. We use a CNN to localize in a topological map, and a conditional VAE to output depth for a camera image, conditional on this topological location estimation. We demonstrate the effectiveness of our monocular localization and depth estimation system on simulated and real datasets.

show abstract

“…On the other side of the spectrum, various approaches have focused on the perception task, employing custom motion planning schemes to determine the robot's action based on the perception output. In [19], for example, a depth map is predicted for each monocular image captured by the drone's on-board camera, using a CNN trained on RGB-D data. Then, a deterministic arbitration scheme is employed to steer the UAV away from obstacles by controlling its angle on two rotational degrees of freedom (DoF), based on the generated depth map.…”

Section: Related Workmentioning

confidence: 99%

Learning to Fly by MySelf: A Self-Supervised CNN-Based Approach for Autonomous Navigation

Kouris

Bouganis

2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Nowadays, Unmanned Aerial Vehicles (UAVs) are becoming increasingly popular facilitated by their extensive availability. Autonomous navigation methods can act as an enabler for the safe deployment of drones on a wide range of real-world civilian applications. In this work, we introduce a self-supervised CNN-based approach for indoor robot navigation. Our method addresses the problem of real-time obstacle avoidance, by employing a regression CNN that predicts the agent's distance-to-collision in view of the raw visual input of its on-board monocular camera. The proposed CNN is trained on our custom indoor-flight dataset which is collected and annotated with real-distance labels, in a self-supervised manner using external sensors mounted on an UAV. By simultaneously processing the current and previous input frame, the proposed CNN extracts spatio-temporal features that encapsulate both static appearance and motion information to estimate the robot's distance to its closest obstacle towards multiple directions. These predictions are used to modulate the yaw and linear velocity of the UAV, in order to navigate autonomously and avoid collisions. Experimental evaluation demonstrates that the proposed approach learns a navigation policy that achieves high accuracy on real-world indoor flights, outperforming previously proposed methods from the literature.

show abstract

CNN-based single image obstacle avoidance on a quadrotor

Cited by 64 publications

References 11 publications

Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge

Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge

GEN-SLAM: Generative Modeling for Monocular Simultaneous Localization and Mapping

Learning to Fly by MySelf: A Self-Supervised CNN-Based Approach for Autonomous Navigation

Contact Info

Product

Resources

About