A Dueling-DDPG Architecture for Mobile Robots Path Planning Based on Laser Range Findings

Zhao, Panpan; Zheng, Jinfang; Zhou, Qinglin; Lyu, Chen; Lyu, Lei

doi:10.1007/978-3-030-89188-6_12

Cited by 3 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on this research, Jesus et al [ 27 ] applied DDPG to the navigation of mobile robots in a virtual environment. Zhao et al [ 28 ] proposed the D‐DDPG, which integrates a dueling network into the critic network to improve the estimation accuracy of the Q ‐value. Gong et al [ 29 ] and Zhou et al [ 30 ] introduced long short‐term memory (LSTM) into the DDPG to achieve long‐term capability in mapless navigation.…”

Section: Related Studiesmentioning

confidence: 99%

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Xue,

Zhang,

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Real‐time path planning in unknown dynamic environments is a significant challenge for mobile robots. Many researchers have attempted to solve this problem by introducing deep reinforcement learning, which trains agents through interaction with their environments. A method called BOAE‐DDPG, which combines the novel bidirectional obstacle avoidance enhancement (BOAE) mechanism with the deep deterministic policy gradient (DDPG) algorithm, is proposed to enhance the learning ability of obstacle avoidance. Inspired by the analysis of the reaction advantage in dynamic psychology, the BOAE mechanism focuses on obstacle‐avoidance reactions from the state and action. The cross‐attention mechanism is incorporated to enhance the attention to valuable obstacle‐avoidance information. Meanwhile, the obstacle‐avoidance behavioral advantage is separately estimated using the modified dueling network. Based on the learning goals of the mobile robot, new assistive reward factors are incorporated into the reward function to promote learning and convergence. The proposed method is validated through several experiments conducted using the simulation platform Gazebo. The results show that the proposed method is suitable for path planning tasks in unknown environments and has an excellent obstacle‐avoidance learning capability.

show abstract

Section: Related Studiesmentioning

confidence: 99%

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Xue,

Zhang,

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…The results [25] revealed that the adopted method was effective in complex environments such as small offices and warehouses, which are typical. The results reported in [24,25] demonstrated that designing a dense reward system suitable for each complex environment is often difficult. Another study [26] implemented path planning for mobile robots in various environments using Mixed Noise-LSTM-DDPG (MN-LSTM-DDPG).…”

Section: Introductionmentioning

confidence: 97%

“…Another study [25] involved path planning for mobile robots in various environments using a dueling DDPG architecture and a dense reward system. The results [25] revealed that the adopted method was effective in complex environments such as small offices and warehouses, which are typical. The results reported in [24,25] demonstrated that designing a dense reward system suitable for each complex environment is often difficult.…”

Section: Introductionmentioning

confidence: 99%

Deep Deterministic Policy Gradient-Based Autonomous Driving for Mobile Robots in Sparse Reward Environments

Park

Lee

Hong

et al. 2022

Sensors

View full text Add to dashboard Cite

In this paper, we propose a deep deterministic policy gradient (DDPG)-based path-planning method for mobile robots by applying the hindsight experience replay (HER) technique to overcome the performance degradation resulting from sparse reward problems occurring in autonomous driving mobile robots. The mobile robot in our analysis was a robot operating system-based TurtleBot3, and the experimental environment was a virtual simulation based on Gazebo. A fully connected neural network was used as the DDPG network based on the actor–critic architecture. Noise was added to the actor network. The robot recognized an unknown environment by measuring distances using a laser sensor and determined the optimized policy to reach its destination. The HER technique improved the learning performance by generating three new episodes with normal experience from a failed episode. The proposed method demonstrated that the HER technique could help mitigate the sparse reward problem; this was further corroborated by the successful autonomous driving results obtained after applying the proposed method to two reward systems, as well as actual experimental results.

show abstract

“…Deep learning has apparent advantages in solving complex tasks and processing high-dimensional data, but it has a limitation in capturing dynamic user preferences timely [2]. Reinforcement learning algorithms can simulate complex user-item interaction processes, and have been successfully applied in recommendation systems and path planning [3]. Deep reinforcement learning algorithms combine the benefits of deep learning and reinforcement learning and are usually used to construct high-dimensional and continuous action space models.…”

Section: Introductionmentioning

confidence: 99%

Trust Recommendation Based on Deep Deterministic Strategy Gradient Algorithm

Tong

2022

IEEE Access

View full text Add to dashboard Cite

Trust recommendation is a vital recommendation system application based on social networks. It can recommend items based on the trust between users, which can alleviate data sparseness and enhance the interpretability of results. A large number of recommendation algorithms have been proposed, but most of them believe that trust is fixed, ignoring the changes of trust in the process of interaction. In addition, deep learning models are good at solving complex tasks and processing high-dimensional data, and they can model recommendation algorithms, but they are insufficient in capturing changes in user preferences timely. Therefore, given the shortcomings of existing researches, we propose a DDPG-TR algorithm based on deep reinforcement learning to capture the changes in user preferences and update the trust between users. The algorithm uses deep deterministic policy gradient algorithm DDPG to model the user-item interaction process. Firstly, we improve a state representative structure to express the user's state, which is convenient to capture the changes in user preferences. Then, when the user accepts the recommendation, algorithm combines trust and similar information to predict item score, as well as calculates the difference of the score. Finally, Agent gets the score feedback and uses the difference to update trust. Experiments are conducted on three datasets, and they are verified that the DDPG-TR algorithm can provide more accurate recommendation results, compared with other recommendation algorithms.

show abstract

A Dueling-DDPG Architecture for Mobile Robots Path Planning Based on Laser Range Findings

Cited by 3 publications

References 25 publications

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Bidirectional Obstacle Avoidance Enhancement‐Deep Deterministic Policy Gradient: A Novel Algorithm for Mobile‐Robot Path Planning in Unknown Dynamic Environments

Deep Deterministic Policy Gradient-Based Autonomous Driving for Mobile Robots in Sparse Reward Environments

Trust Recommendation Based on Deep Deterministic Strategy Gradient Algorithm

Contact Info

Product

Resources

About