Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projection

Abstract

We propose a Model Predictive Control (MPC) method for collision-free navigation that uses amortized variational inference to approximate the distribution of optimal control sequences by training a normalizing flow conditioned on the start, goal and environment. This representation allows us to learn a distribution that accounts for both the dynamics of the robot and complex obstacle geometries. We can then sample from this distribution to produce control sequences which are likely to be both goal-directed and collision-free as part of our proposed FlowMPPI sampling-based MPC method. However, when deploying this method, the robot may encounter an out-of-distribution (OOD) environment, i.e. one which is radically different from those used in training. In such cases, the learned flow cannot be trusted to produce low-cost control sequences. To generalize our method to OOD environments we also present an approach that performs projection on the representation of the environment as part of the MPC process. This projection changes the environment representation to be more in-distribution while also optimizing trajectory quality in the true environment. Our simulation results on a 2D doubleintegrator and a 3D 12DoF underactuated quadrotor suggest that FlowMPPI with projection outperforms state-of-the-art MPC baselines on both in-distribution and OOD environments, including OOD environments generated from real-world data.

ReDUCE: Reformulation of Mixed Integer Programs Using Data from Unsupervised Clusters for Learning Efficient Strategies

“…On the other hand, modern learning techniques such as neural networks can learn an embedding of a larger set of parameters that maps to the solutions [16]. One advantage of using neural network methods is the capability to generalize to out-of-distribution situations that are not included in the training set [29].…”

Hierarchical Subgoal Generation from Language Instruction for Robot Task Planning

“…Learning a prior distribution of actions and subgoals has been used to speed up MPC and accomplish complex tasks. Power and Berenson [25] leverage normalizing flow for modeling the action distributions. Wang and Ba [26] use a policy network to initialize the action sequences for MPC.…”

Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systems

“…Of particular relevance to our framework are methods that combine principled control strategies with learned components in a hierarchical way. Examples include using LQR control in the inner problem with learnable cost and dynamics (Tamar et al, 2017;Amos et al, 2018;Agrawal et al, 2019b), learning sampling distributions in planning and control (Ichter et al, 2018;Power & Berenson, 2022;Amos & Yarats, 2020), or learning optimization strategies or goals for optimization-based control (Sacks & Boots, 2022;Xiao et al, 2022;Metz et al, 2019;2022;Lew et al, 2022).…”