Nonlinear Uncertainty Control with Iterative Covariance Steering

Ridderhof, Jack; Okamoto, K.; Tsiotras, Panagiotis

doi:10.1109/cdc40024.2019.9029993

Cited by 44 publications

(23 citation statements)

References 24 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, input constraints for discrete and continuous time models have been considered in [14,183]. In [207,251], a nonlinear covariance control problem was studied by iteratively solving an approximate linearized problem and by differential dynamic programming, respectively.…”

Section: Stochastic Control and General Bridge Problemsmentioning

confidence: 99%

Stochastic Control Liaisons: Richard Sinkhorn Meets Gaspard Monge on a Schrödinger Bridge

Chen¹,

Georgiou²,

Pavon³

2021

SIAM Rev.

View full text Add to dashboard Cite

Schr\" odinger studied a hot gas Gedankenexperiment (an instance of large deviations of the empirical distribution). Schr\" odinger's problem represents an early example of a fundamental inference method, the so-called maximum entropy method, having roots in Boltzmann's work and being developed in subsequent years by Jaynes, Burg, Dempster, and Csisz\' ar. The problem, known as the Schr\" odinger bridge problem (SBP) with ``uniform"" prior, was more recently recognized as a regularization of the Monge--Kantorovich optimal mass transport (OMT) problem, leading to effective computational schemes for the latter. Specifically, OMT with quadratic cost may be viewed as a zerotemperature limit of the problem posed by Schr\" odinger in the early 1930s. The latter amounts to minimization of Helmholtz's free energy over probability distributions that are constrained to possess two given marginals. The problem features a delicate compromise, mediated by a temperature parameter, between minimizing the internal energy and maximizing the entropy. These concepts are central to a rapidly expanding area of modern science dealing with the so-called Sinkhorn algorithm, which appears as a special case of an algorithm first studied in the more challenging continuous space setting by the French analyst Robert Fortet in 1938--1940 specifically for Schr\" odinger bridges. Due to the constraint on end-point distributions, dynamic programming is not a suitable tool to attack these problems. Instead, Fortet's iterative algorithm and its discrete counterpart, the Sinkhorn iteration, permit computation of the optimal solution by iteratively solving the so-called Schr\" odinger system. Convergence of the iteration is guaranteed by contraction along the steps in suitable metrics, such as Hilbert's projective metric.In both the continuous as well as the discrete time and space settings, stochastic control provides a reformulation of and a context for the dynamic versions of general Schr\" odinger bridge problems and of their zero-temperature limit, the OMT problem. These problems, in turn, naturally lead to steering problems for flows of one-time marginals which represent a new paradigm for controlling uncertainty. The zero-temperature problem in the continuous-time and space setting turns out to be the celebrated Benamou--Brenier characterization of the McCann displacement interpolation flow in OMT. The formalism and techniques behind these control problems on flows of probability distributions have attracted significant attention in recent years as they lead to a variety of new applications in spacecraft guidance, control of robot or biological swarms, sensing, active cooling, and network routing as well as in computer and data science. This multifaceted and versatile framework, intertwining SBP and OMT, provides the substrate for the historical and technical overview \ast

show abstract

Section: Stochastic Control and General Bridge Problemsmentioning

confidence: 99%

Stochastic Control Liaisons: Richard Sinkhorn Meets Gaspard Monge on a Schrödinger Bridge

Chen¹,

Georgiou²,

Pavon³

2021

SIAM Rev.

View full text Add to dashboard Cite

show abstract

“…Equations ( 8) and ( 9) constitute a composite system where the PDE ( 8) is distributedly driven by a collection of SDEs (9). To determine the control inputs u i , consider the following augmented Lyapunov function:…”

Section: Backstepping Density Controlmentioning

confidence: 99%

“…Early efforts on the density control of PDEs tend to adopt an optimal control formulation, which relies on expensive numerical computation of the optimality conditions and usually only generates open-loop control [6], [7]. Closed-loop optimal density control is studied in [8], [9] by establishing a link between density control and Schrödinger Bridge problems. However, except for the linear case which adopts closed-form solutions, numerically solving the associated Schrödinger Bridge problem also suffers from the curse of dimensionality.…”

Section: Introductionmentioning

confidence: 99%

Backstepping Mean-Field Density Control for Large-Scale Heterogeneous Nonlinear Stochastic Systems

Zheng¹,

Han²,

Lin³

2021

Preprint

View full text Add to dashboard Cite

This work studies the problem of controlling the probability density of large-scale stochastic systems, which has applications in various fields such as swarm robotics. Recently, there is a growing amount of literature that employs partial differential equations (PDEs) to model the density evolution and uses density feedback to design control laws which, by acting on individual systems, stabilize their density towards to a target profile. In spite of its stability property and computational efficiency, the success of density feedback relies on assuming the systems to be homogeneous first-order integrators (plus white noise) and ignores higher-order dynamics, making it less applicable in practice. In this work, we present a backstepping design algorithm that extends density control to heterogeneous and higher-order stochastic systems in strict-feedback forms. We show that the strict-feedback form in the individual level corresponds to, in the collective level, a PDE (of densities) distributedly driven by a collection of heterogeneous stochastic systems. The presented backstepping design then starts with a density feedback design for the PDE, followed by a sequence of stabilizing design for the remaining stochastic systems. We present a candidate control law with stability proof and apply it to nonholonomic mobile robots. A simulation is included to verify the effectiveness of the algorithm.

show abstract

“…In contrast with standard LQG control where the final state covariance is indirectly controlled, covariance steering [17] aims at driving the final state mean and covariance to specific prescribed targets. While the first contributions [15,17,33] in the area focused on the steady-state (infinite-horizon) covariance control problem, recently, covariance steering has also been formulated in a finite-horizon control setting, under continuous [2,10,11,12,16] and discrete-time [3,4,7,8] linear dynamics, as well as for systems with partial state information [5,18,27] and nonlinear dynamics [26,31,34].…”

Section: Introductionmentioning

confidence: 99%

Distributed Covariance Steering with Consensus ADMM for Stochastic Multi-Agent Systems

D.¹,

Tsolovikos²,

Bakolas³

et al. 2021

Robotics: Science and Systems XVII

View full text Add to dashboard Cite

In this paper, we address the problem of steering a team of agents under stochastic linear dynamics to prescribed final state means and covariances. The agents operate in a common environment where inter-agent constraints may also be present. In order for our method to be scalable to largescale systems and computationally efficient, we approach the problem in a distributed control framework using the Alternating Direction Method of Multipliers (ADMM). Each agent solves its own covariance steering problem in parallel, while additional copy variables for its closest neighbors are introduced to ensure that the inter-agent constraints will be satisfied. The inclusion of these additional variables creates a requirement for consensus between original and copy variables that involve the same agent. For this reason, we employ a variation of ADMM for consensus optimization. Simulation results on multi-vehicle systems under uncertainty with collision avoidance constraints illustrate the effectiveness of our algorithm. The substantially improved scalability of our distributed approach with respect to the number of agents is also demonstrated, in comparison with an equivalent centralized scheme.

show abstract

Nonlinear Uncertainty Control with Iterative Covariance Steering

Cited by 44 publications

References 24 publications

Stochastic Control Liaisons: Richard Sinkhorn Meets Gaspard Monge on a Schrödinger Bridge

Stochastic Control Liaisons: Richard Sinkhorn Meets Gaspard Monge on a Schrödinger Bridge

Backstepping Mean-Field Density Control for Large-Scale Heterogeneous Nonlinear Stochastic Systems

Distributed Covariance Steering with Consensus ADMM for Stochastic Multi-Agent Systems

Contact Info

Product

Resources

About