Robust Trajectory Optimization: A Cooperative Stochastic Game Theoretic Approach

Pan, Yangdong; Theodorou, Evangelos A.; Bakshi, Kaivalya

doi:10.15607/rss.2015.xi.029

Cited by 9 publications

(17 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Iterative methods via linearization have been widely used in real-time OCP (Pan et al, 2015;Tassa et al, 2014). We adopt a similar methodology for its computational efficiency and algorithmic connection to existing training methods (shown later).…”

Section: Iterative Update Via Linearizationmentioning

confidence: 99%

“…In other words, we can divide the layer's weight (or player's action) into multiple parts, so that the MPDG framework remains applicable. Interestingly, the transformation of this kind resembles game-theoretic robust optimal control (Pan et al, 2015;Sun et al, 2018), where the controller (or player in our context) models external disturbances with fictitious agents, in order to enhance the robustness or convergence of the optimization process.…”

Section: Game-theoretic Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Game Theoretic Neural Optimizer

Liu,

Chen,

Theodorou

2021

Preprint

View full text Add to dashboard Cite

The connection between training deep neural networks (DNNs) and optimal control theory (OCT) has attracted considerable attention as a principled tool of algorithmic design. Despite few attempts being made, they have been limited to architectures where the layer propagation resembles a Markovian dynamical system. This casts doubts on their flexibility to modern networks that heavily rely on non-Markovian dependencies between layers (e.g. skip connections in residual networks). In this work, we propose a novel dynamic game perspective by viewing each layer as a player in a dynamic game characterized by the DNN itself. Through this lens, different classes of optimizers can be seen as matching different types of Nash equilibria, depending on the implicit information structure of each (p)layer. The resulting method, called Dynamic Game Theoretic Neural Optimizer (DGNOpt), not only generalizes OCTinspired optimizers to richer network class; it also motivates a new training principle by solving a multi-player cooperative game. DGNOpt shows convergence improvements over existing methods on image classification datasets with residual networks. Our work marries strengths from both OCT and game theory, paving ways to new algorithmic opportunities from robust optimal control and bandit-based optimization.

show abstract

Section: Iterative Update Via Linearizationmentioning

confidence: 99%

Section: Game-theoretic Applicationsmentioning

confidence: 99%

Dynamic Game Theoretic Neural Optimizer

Liu,

Chen,

Theodorou

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Several robust variants of differential dynamic programming [11] have been proposed for solving worst-case minimax problems [27], risk-sensitive optimizations for stochastic systems [6], and cooperative stochastic games [29]. Like the algorithm proposed in this paper, these methods consider system responses under linear feedback, but they use different robustness metrics and lack the ability to explicitly incorporate bounds on disturbances.…”

Section: Related Workmentioning

confidence: 99%

“…This paper builds on previous work on robust motion planning based on direct trajectory optimization [26,3] and differential dynamic programming (DDP) [27,6,29]. Robust motion planning algorithms often differ in the precise notion of robustness that they seek to optimize.…”

Section: Introductionmentioning

confidence: 99%

DIRTREL: Robust Trajectory Optimization with Ellipsoidal Disturbances and LQR Feedback

Manchester

Kuindersma

2017

Robotics: Science and Systems XIII

View full text Add to dashboard Cite

Abstract-Many critical robotics applications require robustness to disturbances arising from unplanned forces, state uncertainty, and model errors. Motion planning algorithms that explicitly reason about robustness require a coupling of trajectory optimization and feedback design, where the system's closedloop response to bounded disturbances is optimized. Due to the often-heavy computational demands of solving such problems, the practical application of robust trajectory optimization in robotics has so far been limited. We derive a tractable robust optimization algorithm that combines direct transcription with linear-quadratic control design to reason about closed-loop responses to disturbances. In the case of ellipsoidal disturbance sets, the state and input deviations along a nominal trajectory can be computed locally in closed form, thus allowing for fast evaluations of robust cost and constraint functions. The resulting algorithm, called DIRTREL, is an extension of classical direct transcription that demonstrably improves tracking performance over non-robust formulations while incurring only a modest increase in computational cost. We evaluate the algorithm in several simulated robot control tasks.

show abstract

“…The MFG case is further complicated due to the fully coupled nature of the HJB-FP system ( [7], [8], [9]). The first [10] and second ( [11], [12]) order forward-backward SDE (FBSDE) [1] framework has been applied to obtain algorithms for optimal control of dynamics with nonlinear drift and state multiplicative noise, but not in the case of control multiplicative Gaussian or the general case of non-Gaussian excitation [13].…”

Section: Introductionmentioning

confidence: 99%

Open-loop Deterministic Density Control of Marked Jump Diffusions

Bakshi,

Theodorou

2020

Preprint

View full text Add to dashboard Cite

The standard practice in modeling dynamics and optimal control of a large population, ensemble, multi-agent system represented by it's continuum density, is to model individual decision making using local feedback information. In comparison to a closed-loop optimal control scheme, an open-loop strategy, in which a centralized controller broadcasts identical control signals to the ensemble of agents, mitigates the computational and infrastructure requirements for such systems. This work considers the open-loop, deterministic and optimal control synthesis for the density control of agents governed by marked jump diffusion stochastic diffusion equations. The density evolves according to a forward-intime Chapman-Kolmogorov partial integro-differential equation and the necessary optimality conditions are obtained using the infinite dimensional minimum principle (IDMP). We establish the relationship between the IDMP and the dynamic programming principle as well as the IDMP and stochastic dynamic programming for the synthesized controller. Using the linear Feynman-Kac lemma, a sampling-based algorithm to compute the control is presented and demonstrated for agent dynamics with non-affine and nonlinear drift as well as noise terms.

show abstract

Robust Trajectory Optimization: A Cooperative Stochastic Game Theoretic Approach

Cited by 9 publications

References 28 publications

Dynamic Game Theoretic Neural Optimizer

Dynamic Game Theoretic Neural Optimizer

DIRTREL: Robust Trajectory Optimization with Ellipsoidal Disturbances and LQR Feedback

Open-loop Deterministic Density Control of Marked Jump Diffusions

Contact Info

Product

Resources

About