Deep Deterministic Policy Gradient-Based Autonomous Driving for Mobile Robots in Sparse Reward Environments

Abstract

In this paper, we propose a deep deterministic policy gradient (DDPG)-based path-planning method for mobile robots by applying the hindsight experience replay (HER) technique to overcome the performance degradation resulting from sparse reward problems occurring in autonomous driving mobile robots. The mobile robot in our analysis was a robot operating system-based TurtleBot3, and the experimental environment was a virtual simulation based on Gazebo. A fully connected neural network was used as the DDPG network based on the actor–critic architecture. Noise was added to the actor network. The robot recognized an unknown environment by measuring distances using a laser sensor and determined the optimized policy to reach its destination. The HER technique improved the learning performance by generating three new episodes with normal experience from a failed episode. The proposed method demonstrated that the HER technique could help mitigate the sparse reward problem; this was further corroborated by the successful autonomous driving results obtained after applying the proposed method to two reward systems, as well as actual experimental results.

Inspection Robot Navigation Based on Improved TD3 Algorithm

“…The network model outputs continuous linear and angular velocities to control the robot’s forward movement and steering. The and the previous action serve as state input, enabling the neural network to gauge the robot’s speed and its distance from the target [ 34 ]. It is pertinent to highlight that represents the Euclidean distance between the target coordinates and the current robot coordinates , which is defined as follows: …”

Implementing Deep Reinforcement Learning in Autonomous Control Systems

“…They test and validate their approach using the CARLA simulator [25]. However, Park et al, apply the deep deterministic policy gradient (DDPG) path-planning method for mobile robots using Gazebo simulator [26].…”

Autonomous Driving of Mobile Robots in Dynamic Environments Based on Deep Deterministic Policy Gradient: Reward Shaping and Hindsight Experience Replay

“…The technology applied in this study was previously introduced in the authors’ earlier work [ 23 ]. In [ 23 ], a method integrating the HER technique to assist in finding the optimal policy was proposed and demonstrated its effectiveness in both simulation and real-world environments without obstacles.…”

“…The agent was trained in a simple driving environment within the simulation. We demonstrated that the proposed method operates effectively in both simulated and actual environments [ 23 ]. The HER has also been widely applied in the fields of mobile robotics and robot arm control [ 24 , 25 , 26 , 27 , 28 ].…”