Open-loop Deterministic Density Control of Marked Jump Diffusions

Abstract

The standard practice in modeling dynamics and optimal control of a large population, ensemble, multi-agent system represented by it's continuum density, is to model individual decision making using local feedback information. In comparison to a closed-loop optimal control scheme, an open-loop strategy, in which a centralized controller broadcasts identical control signals to the ensemble of agents, mitigates the computational and infrastructure requirements for such systems. This work considers the open-loop, deterministic and optimal control synthesis for the density control of agents governed by marked jump diffusion stochastic diffusion equations. The density evolves according to a forward-intime Chapman-Kolmogorov partial integro-differential equation and the necessary optimality conditions are obtained using the infinite dimensional minimum principle (IDMP). We establish the relationship between the IDMP and the dynamic programming principle as well as the IDMP and stochastic dynamic programming for the synthesized controller. Using the linear Feynman-Kac lemma, a sampling-based algorithm to compute the control is presented and demonstrated for agent dynamics with non-affine and nonlinear drift as well as noise terms.

Limited-control optimal protocols arbitrarily far from equilibrium

“…We have shown that optimal control theory is a natural and powerful framework for the design and study of thermodynamically optimal protocols. In the spirit of [61], it is our hope that through considering the optimal control of non-equilibrium probability densities considered here and elsewhere [34,36,33], we may better understand how it is that biological systems, which operate far from equilibrium, function efficiently across vastly different length-and time-scales.…”

“…We may then apply Pontryagin's maximum principle, one of OCT's foundational theorems, to yield Hamiltonian partial differential equations whose solutions directly give optimal protocols. We note that the optimal control of fields and stochastic systems has been previously studied within applied mathematics and engineering literature [33,34,35,36,37,38,39,40,41,42], but to our knowledge it has never been used to derive exact optimal work-minimizing protocols in stochastic thermodynamics.…”