Optimized-Weighted-Speedy Q-Learning Algorithm for Multi-UGV in Static Environment Path Planning under Anti-Collision Cooperation Mechanism

Abstract

With the accelerated development of smart cities, the concept of a “smart industrial park” in which unmanned ground vehicles (UGVs) have wide application has entered the industrial field of vision. When faced with multiple tasks and heterogeneous tasks, the task execution efficiency of a single UGV is inefficient, thus the task planning research under multi-UGV cooperation has become more urgent. In this paper, under the anti-collision cooperation mechanism for multi-UGV path planning, an improved algorithm with optimized-weighted-speedy Q-learning (OWS Q-learning) is proposed. The slow convergence speed of the Q-learning algorithm is overcome to a certain extent by changing the update mode of the Q function. By improving the selection mode of learning rate and the selection strategy of action, the relationship between exploration and utilization is balanced, and the learning efficiency of multi-agent in complex environments is improved. The simulation experiments in static environment show that the designed anti-collision coordination mechanism effectively solves the coordination problem of multiple UGVs in the same scenario. In the same experimental scenario, compared with the Q-learning algorithm and other reinforcement learning algorithms, only the OWS Q-learning algorithm achieves the convergence effect, and the OWS Q-learning algorithm has the shortest collision-free path for UGVS and the least time to complete the planning. Compared with the Q-learning algorithm, the calculation time of the OWS Q-learning algorithm in the three experimental scenarios is improved by 53.93%, 67.21%, and 53.53%, respectively. This effectively improves the intelligent development of UGV in smart parks.

Multi-Vehicle Collaborative Planning Technology under Automatic Driving

“…Subsequently, Ref. [81] introduces a novel update method for the Q-function, proposing an enhanced Q-learning algorithm tailored for multi-vehicle cooperative obstacle avoidance in static environments. This refined algorithm facilitates the identification of optimal paths within the same iteration, resulting in shortened final path trajectories.…”

Survey of Methods Applied in Cooperative Motion Planning of Multiple Robots

“…Findings show that MMACO-DE reduces distance and saves time by taking lesser turns in a 3D complex environment. Optimized-weighted-speedy Q-learning (OWS Q-learning) algorithm and a collision avoidance cooperation method are suggested for multiple UGVs [28]. Table 3 summarizes the abovementioned literature in tabular form and provides a comparative analysis of the deployed platforms, applied obstacle avoidance protocols, considered obstacles and evaluation indexes, and finally the resulting performance in each study.…”

Mobile Robot Path Planning Based on Kinematically Constrained A-Star Algorithm and DWA Fusion Algorithm

“…In the SLAM navigation process, SLAM (Simultaneous Localization and Mapping) can help the robot determine its position in an unknown environment and build the corresponding environment map, and path planning can help the robot plan its movement route on the basis of SLAM. Path planning is the key for mobile robots to realize autonomous navigation, which mainly refers to the fact that in an environment with obstacles, mobile robots can find a suitable collision-free path from the starting point to the target point according to certain evaluation criteria [3,4]. Mobile robots are typically used in complex environments where static and dynamic obstacles are present, and the path-planning results have a direct impact on the efficiency and safety of the robot [5].…”