Dynamic path planning via Dueling Double Deep Q-Network (D3QN) with prioritized experience replay

Gök, Mehmet

doi:10.1016/j.asoc.2024.111503

Cited by 4 publications

(1 citation statement)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, a noisy network [22] with factorized Gaussian noise injected into the parameters of the online network is introduced to solve the exploration limitation in the DDQN, which can automatically adjust the randomness of action selection to find a better balance between exploration and exploitation. Additionally, an n-step temporal difference learning mechanism is adopted to alleviate the estimation error of the target Q-value, moving much closer to the real target Q-value [23], coupled with a dueling network structure to further alleviate overestimation issues and improve the stability and accuracy of learning by decomposing the Q-value into a state-value function and an advantage function, allowing the network to better learn the relationship between the state value and action advantage [24,25].…”

Section: Introductionmentioning

confidence: 99%

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Hu,

Zhou,

2024

Electronics

View full text Add to dashboard Cite

Effective real-time autonomous navigation for mobile robots in static and dynamic environments has become a challenging and active research topic. Although the simultaneous localization and mapping (SLAM) algorithm offers a solution, it often heavily relies on complex global and local maps, resulting in significant computational demands, slower convergence rates, and prolonged training times. In response to these challenges, this paper presents a novel algorithm called PER-n2D3QN, which integrates prioritized experience replay, a noisy network with factorized Gaussian noise, n-step learning, and a dueling structure into a double deep Q-network. This combination enhances the efficiency of experience replay, facilitates exploration, and provides more accurate Q-value estimates, thereby significantly improving the performance of autonomous navigation for mobile robots. To further bolster the stability and robustness, meaningful improvements, such as target “soft” updates and the gradient clipping mechanism, are employed. Additionally, a novel and powerful target-oriented reshaping reward function is designed to expedite learning. The proposed model is validated through extensive experiments using the robot operating system (ROS) and Gazebo simulation environment. Furthermore, to more specifically reflect the complexity of the simulation environment, this paper presents a quantitative analysis of the simulation environment. The experimental results demonstrate that PER-n2D3QN exhibits heightened accuracy, accelerated convergence rates, and enhanced robustness in both static and dynamic scenarios.

show abstract

Section: Introductionmentioning

confidence: 99%

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Hu,

Zhou,

2024

Electronics

View full text Add to dashboard Cite

show abstract

Dynamic path planning of autonomous bulldozers using activity-value-optimised bio-inspired neural networks and adaptive cell decomposition

Meng,

Gao,

Wang

et al. 2024

Applied Soft Computing

View full text Add to dashboard Cite

A Comprehensive Review of Intelligent Navigation of Mobile Robots Using Reinforcement Learning with A Comparative Analysis of a modified Q-Learning Method and DQN in Simulated Gym Environment

Khlif

2024

Preprint

View full text Add to dashboard Cite

Purpose: The field of autonomous mobile robots (AMRs) has experienced significant growth in recent years, propelled by advancements in autonomous driving and unmanned aerial vehicles (UAVs). The integration of intelligence into robotic systems necessitates addressing various research challenges, with naviga- tion emerging as a pivotal aspect of mobile robotics. This paper explores the three fundamental questions central to the navigation problem: localization (determin- ing the robot’s position), mapping (creating a representation of the environment), and path planning (determining the optimal route to the destination). The pro- posed solution to the mobile robot navigation problem involves the seamless integration of these three foundational navigation components. Methods: Our comparative analysis between the Q-learning modified method and a deep Q-network (DQN) in simulated gym pathfinding tasks reveals the efficacy of this approach. The modified Q-learning algorithm consistently outperforms DQN, demonstrating its superior ability to navigate complex environments and achieve optimal solutions. The transition from a definite environment to a simulated gym environment serves as a valuable validation of the method’s applicability in real-world scenarios. By rigorously evaluating our algorithm in a controlled setting, we can ensure its robustness and effectiveness across a broader range of applications. Results: In essence, our study establishes the modified Q-learning algorithm as a promising new approach to addressing the exploration-exploitation dilemma in reinforcement learning. Its superior performance in simulated gym environments suggests its potential for real-world applications in various domains, including robotics, autonomous navigation, and game development. Conclusion: The paper furnishes a comprehensive overview of research on autonomous mobile robot navigation. It begins with a succinct introduction to the diverse facets of navigation, followed by an examination of the roles of machine learning and reinforcement learning in the realm of mobile robotics. Subsequently, the paper delves into various path planning techniques. In the end, this paper presents a comparative analysis of two path planning methods for mobile robots: Q-learning with an enhanced exploration strategy and Deep Q-Network (DQN). Through a comprehensive simulation study in a gym environment, the superior performance of the proposed Q-learning approach is firmly established.

show abstract

Dynamic path planning via Dueling Double Deep Q-Network (D3QN) with prioritized experience replay

Cited by 4 publications

References 24 publications

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Dynamic path planning of autonomous bulldozers using activity-value-optimised bio-inspired neural networks and adaptive cell decomposition

A Comprehensive Review of Intelligent Navigation of Mobile Robots Using Reinforcement Learning with A Comparative Analysis of a modified Q-Learning Method and DQN in Simulated Gym Environment

Contact Info

Product

Resources

About