A Soft Actor-Critic Approach for a Blind Walking Hexapod Robot with Obstacle Avoidance

Wang, Lei; Li, Ruiwen; Huangfu, Ziwei; Feng, Yishan; Chen, Yiyang

doi:10.3390/act12100393

Cited by 5 publications

(2 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…X. Wang et al [14] utilized the Soft-max function to enhance the DQN algorithm, thereby improving its adaptability for generating actions for hexapod robots. Additionally, L. Wang et al [15] applied the Soft Actor-Critic algorithm for path planning in hexapod robots in outdoor environments, enhancing their robustness in unstructured environments.…”

Section: Introductionmentioning

confidence: 99%

Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots

Chen,

Wang,

Deng

et al. 2024

Sensors

View full text Add to dashboard Cite

Detecting transportation pipeline leakage points within chemical plants is difficult due to complex pathways, multi-dimensional survey points, and highly dynamic scenarios. However, hexapod robots’ maneuverability and adaptability make it an ideal candidate for conducting surveys across different planes. The path-planning problem of hexapod robots in multi-dimensional environments is a significant challenge, especially when identifying suitable transition points and planning shorter paths to reach survey points while traversing multi-level environments. This study proposes a Particle Swarm Optimization (PSO)-guided Double Deep Q-Network (DDQN) approach, namely, the PSO-guided DDQN (PG-DDQN) algorithm, for solving this problem. The proposed algorithm incorporates the PSO algorithm to supplant the traditional random selection strategy, and the data obtained from this guided approach are subsequently employed to train the DDQN neural network. The multi-dimensional random environment is abstracted into localized maps comprising current and next level planes. Comparative experiments were performed with PG-DDQN, standard DQN, and standard DDQN to evaluate the algorithm’s performance by using multiple randomly generated localized maps. After testing each iteration, each algorithm obtained the total reward values and completion times. The results demonstrate that PG-DDQN exhibited faster convergence under an equivalent iteration count. Compared with standard DQN and standard DDQN, reductions in path-planning time of at least 33.94% and 42.60%, respectively, were observed, significantly improving the robot’s mobility. Finally, the PG-DDQN algorithm was integrated with sensors onto a hexapod robot, and validation was performed through Gazebo simulations and Experiment. The results show that controlling hexapod robots by applying PG-DDQN provides valuable insights for path planning to reach transportation pipeline leakage points within chemical plants.

show abstract

Section: Introductionmentioning

confidence: 99%

Improved Double Deep Q-Network Algorithm Applied to Multi-Dimensional Environment Path Planning of Hexapod Robots

Chen,

Wang,

Deng

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…(17)(18)(19)(20) Legged robots are designed to overcome terrain obstacles and execute movements smoothly, but their main drawback is the low speed of movement. (21)(22)(23) Additionally, in unfamiliar terrain, the balancing control problem during dynamic stepping is complex. (24)(25)(26) Recent research has focused on improving the motion speed of legged robots.…”

Section: Introductionmentioning

confidence: 99%