Hierarchical Free Gait Motion Planning for Hexapod Robots Using Deep Reinforcement Learning

Wang, Xinpeng; Fu, Huiqiao; Deng, Guizhou; Liu, Canghai; Tang, Kaiqiang; Chen, Chunlin

doi:10.1109/tii.2023.3240758

Cited by 6 publications

(1 citation statement)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, some researchers have employed reinforcement learning algorithms for path planning in hexapod robots. X. Wang et al [14] utilized the Soft-max function to enhance the DQN algorithm, thereby improving its adaptability for generating actions for hexapod robots. Additionally, L. Wang et al [15] applied the Soft Actor-Critic algorithm for path planning in hexapod robots in outdoor environments, enhancing their robustness in unstructured environments.…”

Section: Introductionmentioning

confidence: 99%

Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots

Chen,

Wang,

Deng

et al. 2024

Sensors

View full text Add to dashboard Cite

Detecting transportation pipeline leakage points within chemical plants is difficult due to complex pathways, multi-dimensional survey points, and highly dynamic scenarios. However, hexapod robots’ maneuverability and adaptability make it an ideal candidate for conducting surveys across different planes. The path-planning problem of hexapod robots in multi-dimensional environments is a significant challenge, especially when identifying suitable transition points and planning shorter paths to reach survey points while traversing multi-level environments. This study proposes a Particle Swarm Optimization (PSO)-guided Double Deep Q-Network (DDQN) approach, namely, the PSO-guided DDQN (PG-DDQN) algorithm, for solving this problem. The proposed algorithm incorporates the PSO algorithm to supplant the traditional random selection strategy, and the data obtained from this guided approach are subsequently employed to train the DDQN neural network. The multi-dimensional random environment is abstracted into localized maps comprising current and next level planes. Comparative experiments were performed with PG-DDQN, standard DQN, and standard DDQN to evaluate the algorithm’s performance by using multiple randomly generated localized maps. After testing each iteration, each algorithm obtained the total reward values and completion times. The results demonstrate that PG-DDQN exhibited faster convergence under an equivalent iteration count. Compared with standard DQN and standard DDQN, reductions in path-planning time of at least 33.94% and 42.60%, respectively, were observed, significantly improving the robot’s mobility. Finally, the PG-DDQN algorithm was integrated with sensors onto a hexapod robot, and validation was performed through Gazebo simulations and Experiment. The results show that controlling hexapod robots by applying PG-DDQN provides valuable insights for path planning to reach transportation pipeline leakage points within chemical plants.

show abstract

Section: Introductionmentioning

confidence: 99%