Optimal Control in Large Stochastic Multi-agent Systems

Abstract

We study optimal control in large stochastic multi-agent systems in continuous space and time. We consider multi-agent systems where agents have independent dynamics with additive noise and control. The goal is to minimize the joint cost, which consists of a state dependent term and a term quadratic in the control. The system is described by a mathematical model, and an explicit solution is given. We focus on large systems where agents have to distribute themselves over a number of targets with minimal cost. In such a setting the optimal control problem is equivalent to a graphical model inference problem. Exact inference will be intractable, and we use the mean field approximation to compute accurate approximations of the optimal controls. We conclude that near to optimal control in large stochastic multi-agent systems is possible with this approach.

Stochastic optimal enhancement of distributed formation control using Kalman smoothers

“…In particular, the relations between the solutions to optimal control PDEs and the probability distribution of stochastic differential equations [19][20][21] allow certain stochastic optimal control problems to be written as an estimation problem on the distribution of optimal trajectories in continuous state space in a manner known as the path integral (PI) approach. 22-26 (1) Related works incorporating the PI framework for multi-agent systems [29][30][31][32] designed control for systems in which agents cooperatively compute their control from a marginalization of the joint probability distribution of the group's trajectory. In this paper (2) we develop a method by which agents independently compute their controls without explicit communication.…”