ME‐MADDPG: An efficient learning‐based motion planning method for multiple agents in complex environments

Abstract

Developing efficient motion policies for multiagents is a challenge in a decentralized dynamic situation, where each agent plans its own paths without knowing the policies of the other agents involved. This paper presents an efficient learning‐based motion planning method for multiagent systems. It adopts the framework of multiagent deep deterministic policy gradient (MADDPG) to directly map partially observed information to motion commands for multiple agents. To improve the efficiency of MADDPG in sample utilization, so as to train more brilliant agents that can adapt to more complex environments, a strategy named mixed experience (ME) is introduced to MADDPG, and this has led to our proposed ME‐MADDPG algorithm. The novel ME strategy can be embodied into three specific mechanisms: (1) an artificial potential field‐based sample generator to produce high‐quality samples in the early training stage; (2) a dynamic mixed sampling strategy to mix the training data from different sources with a variable proportion; (3) a delayed learning skill to stabilize the training of the multiple agents. A series of experiments have been conducted to verify the performance of the proposed ME‐MADDPG algorithm, and it has been demonstrated that, compared with MADDPG, the proposed algorithm can significantly improve the convergence speed and convergence effect in the training process, and it has also shown better efficiency and better adaptability in complex dynamic environments while it is used for multiagent motion planning applications.

On Multi-Agent Deep Deterministic Policy Gradients and their Explainability for SMARTS Environment

“…There has been a considerable extension to the MADDPG paradigm where recurrent DPGs were used for complex environments like Cognitive Electronic Warfare [18] and Partially Observable Environments for Communication systems [19]. A mixed environment approach was taken for complex environments using MADDPG [20], whereas Decomposed Approach was introduced for learning multi-agent policies for UAV clusters to build a connected communication network [21]. Further, these set of algorithms were also discussed in the context of working with smart grids for edge technology [22] and have shown to perform considerably well when compared to the other state-of-the-art.…”

Decentralized Navigation with Optimality for Multiple Holonomic Agents in Simply Connected Workspaces

“…The CM relies on Reinforcement Learning (RL)-based methods that use iterative algorithms to converge in an optimal navigation policy [30][31][32]. It is common in MAS to use methods based on Deep Reinforcement Learning (DRL), which is a powerful tool that combines neural networks and RL algorithms that allow each agent to learn from its interactions with the environment [33][34][35][36][37][38]. Despite the effectiveness of the RL-based methods, the main disadvantage in MAS is the computational complexity and abundance of data required to converge to the global policy.…”

Time-Optimal Paths for Simple Cars with Moving Obstacles in the Hamilton-Jacobi Formulation

“…Trajectory generation methods which are rooted in PDE have the advantage that they are easy to implement, entirely interpretable, and can provide theoretical guarantees regarding optimality, robustness, and other concerns. This is to distinguish them from sampling and learning based algorithms (for example [15,16,17,18,19]) which often sacrifice interpretability for efficiency. The main drawbacks of the PDEbased methods are the lack of efficiency and scalability.…”