Stochastic control of systems with control multiplicative noise using second order FBSDEs

Bakshi, Kaivalya; Fan, David D.; Theodorou, Evangelos A.

doi:10.23919/acc.2017.7962990

Cited by 4 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Let us denote the set of initial perturbed densities by S = p(0, x) = p ∞ (x)(1+p(0, x)) p(0, x) ≥ 0, p(0, x) ∈ S 0 . We say the equilibrium stationary density p ∞ (x) of the FP equation ( 6) is asymptotically stable with respect to S if there exists a solution p(t, x) to the perturbation equation (14) such that lim t→+∞ ||p(t, x)|| H = 0.…”

Section: E Stability Analysismentioning

confidence: 99%

“…is the unique H solution to the perturbation equation (14). p ∞ (x) is asymptotically stable with respect to S.…”

Section: E Stability Analysismentioning

confidence: 99%

“…Using these properties yields the ODEs ( 16) with the unique solutions p n (t) = p n (0)e ξnt . Therefore p(t, x) = +∞ n=0 p n (t)Ξ n (x) wherein p n (t) = p n (0)e ξnt is the unique H solution to the perturbation equation (14). Further, p(0, x) ∈ S 0 implies that p 0 (0) = p(0, x), Ξ 0 (x) H = p(0, x), 1 H = 0 implying that p 0 (t) = 0 for all 0 ≤ t, which completes the first part of the proof.…”

Section: E Stability Analysismentioning

confidence: 99%

“…The multiplicative noise severely affects stability and robustness [12] and the optimal control formulation in this case is non-standard due to the non-Gaussianity of the dynamics [9], and most works addressing either aspect in the linear-quadratic regime. In the case of nonlinear dynamics with multiplicative noise, differential dynamic programming [11] and forward backward stochastic differential equations using first [13] and second order schemes ( [14], [15]) have been used to synthesize algorithms to compute the control.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Stabilizing Optimal Density Control of Nonlinear Agents with Multiplicative Noise

Bakshi¹,

Theodorou²,

Grover³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Control of continuous time dynamics with multiplicative noise is a classic topic in stochastic optimal control. This work addresses the problem of designing infinite horizon optimal controls with stability guarantees for large populations of identical, non-cooperative and non-networked agents with multi-dimensional and nonlinear stochastic dynamics excited by multiplicative noise. We provide constraints on the state and control cost functions which guarantee stability of the closed-loop system under the action of the individual optimal controls, for agent dynamics belonging to the the class of reversible diffusion processes. A condition relating the state-dependent control cost and volatility is introduced to prove the stability of the equilibrium density. This condition is a special case of the constraint required to use the path integral Feynman-Kac formula for computing the control. We investigate the connection between the stabilizing optimal control and the path integral formalism, leading us to a control law formulation expressed exclusively in terms of the desired equilibrium density.

show abstract

Section: E Stability Analysismentioning

confidence: 99%

“…is the unique H solution to the perturbation equation (14). p ∞ (x) is asymptotically stable with respect to S.…”

Section: E Stability Analysismentioning

confidence: 99%

Section: E Stability Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Stabilizing Optimal Density Control of Nonlinear Agents with Multiplicative Noise

Bakshi¹,

Theodorou²,

Grover³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The MFG case is further complicated due to the fully coupled nature of the HJB-FP system ( [7], [8], [9]). The first [10] and second ( [11], [12]) order forward-backward SDE (FBSDE) [1] framework has been applied to obtain algorithms for optimal control of dynamics with nonlinear drift and state multiplicative noise, but not in the case of control multiplicative Gaussian or the general case of non-Gaussian excitation [13].…”

Section: Introductionmentioning

confidence: 99%

Open-loop Deterministic Density Control of Marked Jump Diffusions

Bakshi,

Theodorou

2020

Preprint

Self Cite

View full text Add to dashboard Cite

The standard practice in modeling dynamics and optimal control of a large population, ensemble, multi-agent system represented by it's continuum density, is to model individual decision making using local feedback information. In comparison to a closed-loop optimal control scheme, an open-loop strategy, in which a centralized controller broadcasts identical control signals to the ensemble of agents, mitigates the computational and infrastructure requirements for such systems. This work considers the open-loop, deterministic and optimal control synthesis for the density control of agents governed by marked jump diffusion stochastic diffusion equations. The density evolves according to a forward-intime Chapman-Kolmogorov partial integro-differential equation and the necessary optimality conditions are obtained using the infinite dimensional minimum principle (IDMP). We establish the relationship between the IDMP and the dynamic programming principle as well as the IDMP and stochastic dynamic programming for the synthesized controller. Using the linear Feynman-Kac lemma, a sampling-based algorithm to compute the control is presented and demonstrated for agent dynamics with non-affine and nonlinear drift as well as noise terms.

show abstract

Stabilizing Optimal Density Control of Nonlinear Agents with Multiplicative Noise

Bakshi

Theodorou

Grover

2020

2020 59th IEEE Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Stochastic control of systems with control multiplicative noise using second order FBSDEs

Cited by 4 publications

References 20 publications

Stabilizing Optimal Density Control of Nonlinear Agents with Multiplicative Noise

Stabilizing Optimal Density Control of Nonlinear Agents with Multiplicative Noise

Open-loop Deterministic Density Control of Marked Jump Diffusions

Stabilizing Optimal Density Control of Nonlinear Agents with Multiplicative Noise

Contact Info

Product

Resources

About