Deep Deterministic Policy Gradient for Navigation of Mobile Robots in Simulated Environments

Jesus, Junior Costa de; Bottega, Jair Augusto; Cuadros, Marco Antônio de Souza Leite; Gamarra, Daniel Fernando Tello

doi:10.1109/icar46387.2019.8981638

Cited by 35 publications

(17 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Embodied in a neural network, the expected result is an action for the current state. This arrange for the data was explored in Tai et al [7], Jesus et al in [9] and in our previous work [12] for 3D navigation of UAV.…”

Section: Methodsmentioning

confidence: 99%

“…In this context, Deep-RL ended up simplifying the problem by discretizing it [6]. Other approaches have also been exploring continuous control actions in the navigation of mobile robots and achieving interesting results [7], [8], [9]. However, the results are yet limited for autonomous Unmanned Aerial Vehicles (UAVs) [10,11] In our previous work, we proposed the adaptation of two Deep-RL techniques for an UAV in two environments [12] and explored the concept for mobile robots [13].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Grando

Jesus

Drews-Jr

2020

2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Educ

View full text Add to dashboard Cite

This paper presents a novel deep reinforcement learning-based system for 3D mapless navigation for Unmanned Aerial Vehicles (UAVs). Instead of using a image-based sensing approach, we propose a simple learning system that uses only a few sparse range data from a distance sensor to train a learning agent. We based our approaches on two state-of-art double critic Deep-RL models: Twin Delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC). We show that our two approaches manage to outperform an approach based on the Deep Deterministic Policy Gradient (DDPG) technique and the BUG2 algorithm. Also, our new Deep-RL structure based on Recurrent Neural Networks (RNNs) outperforms the current structure used to perform mapless navigation of mobile robots. Overall, we conclude that Deep-RL approaches based on double critic with Recurrent Neural Networks (RNNs) are better suited to perform mapless navigation and obstacle avoidance of UAVs.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Grando

Jesus

Drews-Jr

2020

2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Educ

View full text Add to dashboard Cite

show abstract

“…The study concluded that a mapless motion planner based on the DDPG algorithm can be successfully trained and used to complete the task of navigating to a target. Zhu et al [5], Chen et al [21], Ota et al [22], de Jesus et al [23] and others have already successfully applied Deep-RL approaches to perform mapless navigation-related tasks for terrestrial mobile robots.…”

Section: Related Workmentioning

confidence: 99%

“…1) 3D Navigation Deep Reinforcement Learning Deterministic: The DDPG architecture has been proposed as an extension to the DQN, and it became a founding block for Deep-RL applications of mobile robots in continuous observation spaces [23]. It consists of an actor-critic approach that uses approximation functions to learn continuous space policies.…”

Section: A Deep Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning for Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Medium Transition

Grando¹,

Jesus²,

Kich³

et al. 2021

Preprint

View full text Add to dashboard Cite

Since the application of Deep Q-Learning to the continuous action domain in Atari-like games, Deep Reinforcement Learning (Deep-RL) techniques for motion control have been qualitatively enhanced. Nowadays, modern Deep-RL can be successfully applied to solve a wide range of complex decision-making tasks for many types of vehicles. Based on this context, in this paper, we propose the use of Deep-RL to perform autonomous mapless navigation for Hybrid Unmanned Aerial Underwater Vehicles (HUAUVs), robots that can operate in both, air or water media. We developed two approaches, one deterministic and the other stochastic. Our system uses the relative localization of the vehicle and simple sparse range data to train the network. We compared our approaches with a traditional geometric tracking controller for mapless navigation. Based on experimental results, we can conclude that Deep-RLbased approaches can be successfully used to perform mapless navigation and obstacle avoidance for HUAUVs. Our vehicle accomplished the navigation in two scenarios, being capable to achieve the desired target through both environments, and even outperforming the geometric-based tracking controller on the obstacle-avoidance capability.

show abstract

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Xue,

Zhang,

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Real‐time path planning in unknown dynamic environments is a significant challenge for mobile robots. Many researchers have attempted to solve this problem by introducing deep reinforcement learning, which trains agents through interaction with their environments. A method called BOAE‐DDPG, which combines the novel bidirectional obstacle avoidance enhancement (BOAE) mechanism with the deep deterministic policy gradient (DDPG) algorithm, is proposed to enhance the learning ability of obstacle avoidance. Inspired by the analysis of the reaction advantage in dynamic psychology, the BOAE mechanism focuses on obstacle‐avoidance reactions from the state and action. The cross‐attention mechanism is incorporated to enhance the attention to valuable obstacle‐avoidance information. Meanwhile, the obstacle‐avoidance behavioral advantage is separately estimated using the modified dueling network. Based on the learning goals of the mobile robot, new assistive reward factors are incorporated into the reward function to promote learning and convergence. The proposed method is validated through several experiments conducted using the simulation platform Gazebo. The results show that the proposed method is suitable for path planning tasks in unknown environments and has an excellent obstacle‐avoidance learning capability.

show abstract

Deep Deterministic Policy Gradient for Navigation of Mobile Robots in Simulated Environments

Cited by 35 publications

References 7 publications

Deep Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Deep Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Deep Reinforcement Learning for Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Medium Transition

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Contact Info

Product

Resources

About