A Neural Network Approach Applied to Multi-Agent Optimal Control

Abstract

We propose a neural network approach for solving high-dimensional optimal control problems. In particular, we focus on multi-agent control problems with obstacle and collision avoidance. These problems immediately become highdimensional, even for moderate phase-space dimensions per agent. Our approach fuses the Pontryagin's Maximum Principle and Hamilton-Jacobi-Bellman (HJB) approaches and parameterizes the value function with a neural network. Our approach yields controls in a feedback form for quick calculation and robustness to moderate disturbances to the system.We train our model using the objective function and optimality conditions of the control problem. Therefore, our training algorithm neither involves a data generation phase nor solutions from another algorithm. Our model uses empirically effective HJB penalizers for efficient training. By training on a distribution of initial states, we ensure the controls' optimality is achieved on a large portion of the state-space. Our approach is grid-free and scales efficiently to dimensions where grids become impractical or infeasible. We demonstrate our approach's effectiveness on a 150-dimensional multi-agent problem with obstacles.

Adaptive Uncertainty-Weighted ADMM for Distributed Optimization

“…A natural extension of UQ-ADMM involves its application to more general non-convex problems such as training deep neural networks for classification [20]. In particular, the robustness with respect to the number of splittings could be well-tailored toward training swarm-based multi-agent control models [23,24]. However, convergence guarantees for the non-convex setting is a more difficult task and is a direction we intend to pursue.…”

A Neural Network Approach for High-Dimensional Optimal Control Applied to Multi-Agent Path Finding

“…This paper extends a preliminary conference version of the approach [11] with more extensive and thorough experiments. Specifically, we add experiments where agents swap positions with each other and one involving a nonlinear controlaffine quadcopter with complicated dynamics.…”