Approximately optimal distributed stochastic controls beyond the mean field setting

Abstract

We study high-dimensional stochastic optimal control problems in which many agents cooperate to minimize a convex cost functional. We consider both the full-information problem, in which each agent observes the states of all other agents, and the distributed problem, in which each agent observes only its own state. Our main results are sharp non-asymptotic bounds on the gap between these two problems, measured both in terms of their value functions and optimal states. Along the way, we develop theory for distributed optimal stochastic control in parallel with the classical setting, by characterizing optimizers in terms of an associated stochastic maximum principle and a Hamilton-Jacobi-type equation. By specializing these results to the setting of mean field control, in which costs are (symmetric) functions of the empirical distribution of states, we derive the optimal rate for the convergence problem in the displacement convex regime. Contents 1. Introduction 2. Notation and preliminaries 3. Problem formulations and value functions 4. Near-optimality of distributed controls 5. Application to mean field control 6. Application to heterogeneous doubly stochastic interactions 7. Tradeoffs between convexity and smallness Appendix A. Well-posedness of maximum principle FBSDE References