Schrödinger Approach to Optimal Control of Large-Size Populations

Bakshi, Kaivalya; Fan, David D.; Theodorou, Evangelos A.

doi:10.1109/tac.2020.3007543

Cited by 14 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonetheless, for MASs with massive population, i.e. N → ∞, most of the control schemes and algorithms introduced in this paper will become fruitless, and we may resort to the mean-field theory [9,45,46], which describes MASs by probability density model rather than connected graph model.…”

Section: B Distributed Planning Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems

Wan¹,

Gahlawat²,

Hovakimyan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Distributed algorithms for both discrete-time and continuous-time linearly solvable optimal control (LSOC) problems of networked multi-agent systems (MASs) are investigated in this paper. A distributed framework is proposed to partition the optimal control problem of a networked MAS into several local optimal control problems in factorial subsystems, such that each (central) agent behaves optimally to minimize the joint cost function of a subsystem that comprises a central agent and its neighboring agents, and the local control actions (policies) only rely on the knowledge of local observations. Under this framework, we not only preserve the correlations between neighboring agents, but moderate the communication and computational complexities by decentralizing the sampling and computational processes over the network. For discrete-time systems modeled by Markov decision processes, the joint Bellman equation of each subsystem is transformed into a system of linear equations and solved using parallel programming. For continuous-time systems modeled by Itô diffusion processes, the joint optimality equation of each subsystem is converted into a linear partial differential equation, whose solution is approximated by a path integral formulation and a sample-efficient relative entropy policy search algorithm, respectively. The learned control policies are generalized to solve the unlearned tasks by resorting to the compositionality principle, and illustrative examples of cooperative UAV teams are provided to verify the effectiveness and advantages of these algorithms.

show abstract

Section: B Distributed Planning Algorithmmentioning

confidence: 99%

“…holds everywhere from t to t f on condition that (46) holds for all terminal states Bi , which is guaranteed by (45). Substituting (46) into the continuous-time optimal controller (28), the taskoptimal composite controller ū * i (x i ) can be constructed from the component controllers ū{f} *…”

Section: Generalization With Compositionalitymentioning

confidence: 99%

Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems

Wan¹,

Gahlawat²,

Hovakimyan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…On the other hand, recently, a different approach to solve a dynamical assignment problem using OT theory has attracted much attention [11][12][13]. In this approach, a large population limit is considered, and infinitely many agents are represented as a probability density of the state of a single system.…”

Section: Introductionmentioning

confidence: 99%

Entropic Model Predictive Optimal Transport over Dynamical Systems

Ito¹,

Kashima²

2023

Preprint

View full text Add to dashboard Cite

We consider the optimal control problem of steering an agent population to a desired distribution over an infinite horizon. This is an optimal transport problem over dynamical systems, which is challenging due to its high computational cost. In this paper, by using entropy regularization, we propose Sinkhorn MPC, which is a dynamical transport algorithm integrating model predictive control (MPC) and the so-called Sinkhorn algorithm. The notable feature of the proposed method is that it achieves cost-effective transport in real time by performing control and transport planning simultaneously, which is illustrated in numerical examples. Moreover, under some assumption on iterations of the Sinkhorn algorithm integrated in MPC, we reveal the global convergence property for Sinkhorn MPC thanks to the entropy regularization. Furthermore, focusing on a quadratic control cost, without the aforementioned assumption we show the ultimate boundedness and the local asymptotic stability for Sinkhorn MPC.

show abstract

“…Although several control theoretical frameworks exist for analyzing and controlling swarming behaviors [25,26,27,28], many are not necessarily tailored to fish schooling because the aforementioned works do not explicitly consider the short-distance repulsion, middle-range alignment, and longdistance attraction rule used commonly in the mathematical modeling of the motion of fish schools [15,16,17,18,19]. One exception is the work by Li et al [29], where the authors proposed a method to design decentralized con-trollers utilizing an attraction/alignment/repulsion law for a swarm of mobile agents to achieve a collective group behavior.…”

Section: Introductionmentioning

confidence: 99%

Reduced-Order Model Predictive Control of a Fish Schooling Model

Ogura¹,

Wakamiya²

2023

Preprint

View full text Add to dashboard Cite

We study the problem of model predictive control (MPC) for the fish schooling model proposed by Gautrais et al. (Annales Zoologici Fennici, 2008). The high nonlinearity of the model attributed to its attraction/alignment/repulsion law suggests the need to use MPC for controlling the fish schooling's motion. However, for large schools, the hybrid nature of the law can make it numerically demanding to perform finite-horizon optimizations in MPC. Therefore, this paper proposes reducing the fish schooling model for numerically efficient MPC; the reduction is based on using the weighted average of the directions of individual fish in the school. We analytically show how using the normalized eigenvector centrality of the alignment-interaction network can yield a better reduction by comparing reduction errors. We confirm this finding on the weight and numerical efficiency of the MPC with the reduced-order model by numerical simulations. The proposed reduction allows us to control a school with up to 500 individuals. Further, we confirm that reduction with the normalized eigenvector centrality allows us to improve the control accuracy by factor of five when compared to that using constant weights.

show abstract

Schrödinger Approach to Optimal Control of Large-Size Populations

Cited by 14 publications

References 33 publications

Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems

Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems

Entropic Model Predictive Optimal Transport over Dynamical Systems

Reduced-Order Model Predictive Control of a Fish Schooling Model

Contact Info

Product

Resources

About