A Path Planning Method for Multi-robot Formation Based on Improved Q-Learning

Abstract

This paper studies a path planning method for multiple robots in unknown environment. Multiple robots adopt the leaderfollowing formation method. For the Q-learning algorithm used by the leader robot, the Q-table is initialized by prior information of environment and the idea of filling concave obstacles is proposed. Then the strategy of choosing actions is improved by simulated annealing algorithm, which changes the greedy factor in real time according to the Q-learning. The follower robot uses an improved gravitational potential field method to follow the leader robot. The simulation results show that the improved algorithm is effective and multiple robots can plan an optimum path to reach the destination with this method.

A comprehensive review of the latest path planning developments for multi-robot formation systems

“…Finally, the literature has shown a larger number of papers proposing UGV leader-follower systems compared to USV and UAV systems. Variety path planning techniques were applied, including gridbased [50], potential field [51][52][53], optimization and machine learning [54][55][56][57][58], and other techniques [59][60][61]. Examining localization tasks in uncertain environments is the objective of the authors in ref.…”

“…[52], Voronoi FM method was proposed as a path planning technique. Another path planning category used in leader-follower UGV systems is the optimization and machine learning methods [54][55][56][57]. Q-Learning algorithm was utilized by the authors of [54], while the Integer Programming method was used in ref.…”

Path Planning Techniques for Real-Time Multi-Robot Systems: A Systematic Review

“…Examples of homogeneous systems encompass Unmanned Ground Vehicles (UGVs) [13], Unmanned Aerial Vehicles (UAVs) [14], Unmanned Surface Vehicles (USVs) [15], and Unmanned Underwater Vehicles (UUVs). Examples of heterogeneous systems include Unmanned Aerial Vehicles and Unmanned Ground Vehicle (UAV-UGV) systems [16,17].…”