Online robot guidance and navigation in non-stationary environment with hybrid Hierarchical Reinforcement Learning

Zhou, Ye; Ho, Hann Woei

doi:10.1016/j.engappai.2022.105152

Cited by 8 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, DRL harnesses the efficacy of RL in addressing challenges in sequential decisionmaking [10]. This empowers the agent to gradually learn optimal decision strategies to maximize cumulative rewards or achieve specific goals through interactions with its environment [11,12].…”

Section: Introductionmentioning

confidence: 99%

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Hu,

Zhou,

2024

Electronics

Self Cite

View full text Add to dashboard Cite

Effective real-time autonomous navigation for mobile robots in static and dynamic environments has become a challenging and active research topic. Although the simultaneous localization and mapping (SLAM) algorithm offers a solution, it often heavily relies on complex global and local maps, resulting in significant computational demands, slower convergence rates, and prolonged training times. In response to these challenges, this paper presents a novel algorithm called PER-n2D3QN, which integrates prioritized experience replay, a noisy network with factorized Gaussian noise, n-step learning, and a dueling structure into a double deep Q-network. This combination enhances the efficiency of experience replay, facilitates exploration, and provides more accurate Q-value estimates, thereby significantly improving the performance of autonomous navigation for mobile robots. To further bolster the stability and robustness, meaningful improvements, such as target “soft” updates and the gradient clipping mechanism, are employed. Additionally, a novel and powerful target-oriented reshaping reward function is designed to expedite learning. The proposed model is validated through extensive experiments using the robot operating system (ROS) and Gazebo simulation environment. Furthermore, to more specifically reflect the complexity of the simulation environment, this paper presents a quantitative analysis of the simulation environment. The experimental results demonstrate that PER-n2D3QN exhibits heightened accuracy, accelerated convergence rates, and enhanced robustness in both static and dynamic scenarios.

show abstract

Section: Introductionmentioning

confidence: 99%

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Hu,

Zhou,

2024

Electronics

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the past two decades, numerous coverage optimization and/or control techniques had been developed for addressing a large class of area coverage problems using multiple autonomous agents deployed in two-dimensional spatial environments (Cortes et al 2004;Wang and Hussein 2010;Leonard and Olshevsky 2013;Pimenta et al 2013;. In most cases, autonomous robots are deployed in dynamic or non-stationary environment (Zhou and Ho 2022). It is worth noting that most of the existing technique to address such a large class of area coverage problems in the literature assumed the fact that the communication/data packet sharing among robots/agents is perfect or noise-free.…”

Section: Introductionmentioning

confidence: 99%

Area Coverage Optimization using Networked Mobile Robots with State Estimation

Chism,

Plante,

Miah

2023

FLAIRS

View full text Add to dashboard Cite

In this paper, we present a solution to the area coverage problem using a team of mobile robots with state estimation. A group of autonomous mobile robots is deployed in a two-dimensional area of interest, for example, where communication among robots is limited or noisy as expected in a real life scenario. Each robot estimates its own state (position and orientation) using noisy range and bearing information received from other robots in its operating range. The area of interest is then divided into multiple sub-area using a voronoi tessellation. Using the classical Lloyd's algorithm, each robot employs distributed action command to move towards the centroid of the voronoi cell that it belongs yielding the maximum coverage of the area. Here we emphasize that the network of autonomous robots deployed in the environment is unknown a priori. A set of computer experiments is conducted to validate the fact that the area coverage is still possible under noisy communication among robots deployed in a two-dimensional area.

show abstract

Optimal non-autonomous area coverage control with adaptive reinforcement learning

Soleymani

Miah

Spinello

2023

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

Online robot guidance and navigation in non-stationary environment with hybrid Hierarchical Reinforcement Learning

Cited by 8 publications

References 40 publications

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Area Coverage Optimization using Networked Mobile Robots with State Estimation

Optimal non-autonomous area coverage control with adaptive reinforcement learning

Contact Info

Product

Resources

About