Information-Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving

Abstract

We present an information theoretic approach to stochastic optimal control problems that can be used to derive general sampling based optimization schemes. This new mathematical method is used to develop a sampling based model predictive control algorithm. We apply this information theoretic model predictive control (IT-MPC) scheme to the task of aggressive autonomous driving around a dirt test track, and compare its performance to a model predictive control version of the cross-entropy method.

Acceleration of Gradient-Based Path Integral Method for Efficient Optimal and Inverse Optimal Control

“…The iterative path integral methods [2]- [5] are optimization methods for the stochastic optimal control problem. These methods assume that the system noise t is zeromean Gaussian t ∼ N (0, Σ) with a covariance matrix Σ ∈ R m×m , and suppose a trajectory cost function S(τ ) as the sum of arbitrary state-cost and quadratic control-cost over time time-horizon:…”

“…This section briefly reviews the formulation of stochastic optimal control problem, Williams's iterative path integral methods [2]- [5], and PI-Net [6].…”

“…This approach solves the intrinsic problem of primitive path integral methods that require almost infinite samples for optimal solutions. Moreover, Williams derived different iterative methods [4], [5], which eliminate the affine dynamics constraints on the original path integral framework. This paper focuses on this type of iterative path integral methods.…”

“…First, as it is not necessary to approximate dynamics and cost models with linear and quadratic forms, non linear system dynamics and cost functions can be naturally employed. In [2], [5], highly non linear car dynamics have been employed to successfully control a miniature real vehicle that functions autonomously and with aggressive drifting. Dynamics can also be represented as trainable models, i.e., neural networks, thus allowing to solve model-based reinforcement learning tasks.…”

An Online Learning Approach to Model Predictive Control

“…DMD-MPC is based on a first-order online learning algorithm called dynamic mirror descent (DMD) [14], a generalization of mirror descent [4] for dynamic comparators. We show that several existing MPC algorithms [31,32] are special cases of DMD-MPC, given specific choices of step sizes, loss functions, and regularization. Furthermore, we demonstrate how new MPC algorithms can be derived systematically from DMD-MPC with only mild assumptions on the regularity of the cost function.…”