Enhanced Q-learning algorithm with eligibility traces for safe path planning of unmanned surface vehicles in dynamic unknown environment

Abstract

This study explores the application of a path planning algorithm based on Q-learning and eligibility traces in autonomous task execution for Unmanned Surface Vehicles (USVs). The algorithm aims to provide secure path planning for USVs in dynamic unknown environments, taking into account obstacles, potential threats, and multiple constraints. Initially, a detailed Markov Decision Process (MDP) model was designed. Subsequently, the introduced Q-learning and eligibility trace algorithm demonstrated significant advantages in path planning, utilizing the Upper Confidence Bound (UCB) strategy for action selection. Finally, simulation experiment results indicate that, compared to traditional Q-learning methods, the algorithm can more effectively plan paths for USVs, avoid threat areas, and achieve faster convergence.