TaSIL: Taylor Series Imitation Learning

Pfrommer, Daniel; Zhang, Thomas T. C. K.; Tu, Stephen; Matni, Nikolai

doi:10.48550/arxiv.2205.14812

Cited by 4 publications

(4 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this result does not immediately imply that π and π follow similar trajectories (errors could compound over time, causing π's trajectories to diverge from those of π ). Following the main ideas of Tu et al [46] and Pfrommer et al [31], one can in fact show guarantees in terms of trajectories when the closed-loop system under π is robust in an appropriate sense. See Appendix B the details.…”

Section: Infinite Model Classesmentioning

confidence: 96%

Model Predictive Control via On-Policy Imitation Learning

Ahn¹,

Mhammedi²,

Mania³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we leverage the rapid advances in imitation learning, a topic of intense recent focus in the Reinforcement Learning (RL) literature, to develop new sample complexity results and performance guarantees for data-driven Model Predictive Control (MPC) for constrained linear systems. In its simplest form, imitation learning is an approach that tries to learn an expert policy by querying samples from an expert. Recent approaches to data-driven MPC have used the simplest form of imitation learning known as behavior cloning to learn controllers that mimic the performance of MPC by online sampling of the trajectories of the closed-loop MPC system. Behavior cloning, however, is a method that is known to be data inefficient and suffer from distribution shifts. As an alternative, we develop a variant of the forward training algorithm which is an on-policy imitation learning method proposed by Ross and Bagnell [39]. Our algorithm uses the structure of constrained linear MPC, and our analysis uses the properties of the explicit MPC solution to theoretically bound the number of online MPC trajectories needed to achieve optimal performance. We validate our results through simulations and show that the forward training algorithm is indeed superior to behavior cloning when applied to MPC.

show abstract

Section: Infinite Model Classesmentioning

confidence: 96%

Model Predictive Control via On-Policy Imitation Learning

Ahn¹,

Mhammedi²,

Mania³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Stability of learning-based MPC was established in [2,72] and followed, for nonlinear systems, by efforts on joint learning of the controller and(or) Lyapunov functions [13-15, 21, 22]. In addition, [64,83] have explored how learning-based control affects systems with known Lyapunov functions, [12,23,68] studied learning of stability certificates and stable controllers from data, and [6] developed a provably stable data-driven algorithm based on system measurements and prior system knowledge.…”

Section: Control Design Problems For Hyperbolic Pdes Are Hyperbolicmentioning

confidence: 99%

Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEs

Krstić¹,

Bhan²,

Shi³

2023

Preprint

View full text Add to dashboard Cite

Unlike ODEs, whose models involve system matrices and whose controllers involve vector or matrix gains, PDE models involve functions in those roles-functional coefficients, dependent on the spatial variables, and gain functions dependent on space as well. The designs of gains for controllers and observers for PDEs, such as PDE backstepping, are mappings of system model functions into gain functions. These infinite-dimensional nonlinear operators are given in an implicit form through PDEs, in spatial variables, which need to be solved to determine the gain function for each new functional coefficient of the PDE. The need for solving such PDEs can be eliminated by learning and approximating the said design mapping in the form of a neural operator. Learning the neural operator requires a sufficient number of prior solutions for the design PDEs, offline, as well as the training of the operator. In recent work, we developed the neural operators for PDE backstepping designs for first-order hyperbolic PDEs. Here we extend this framework to the more complex class of parabolic PDEs. The key theoretical question is whether the controllers are still stabilizing, and whether the observers are still convergent, if they employ the approximate functional gains generated by the neural operator. We provide affirmative answers to these questions, namely, we prove stability in closed loop under gains produced by neural operators. We illustrate the theoretical results with numerical tests and publish our code on github. The neural operators are three orders of magnitude faster in generating gain functions than PDE solvers for such gain functions. This opens up the opportunity for the use of this neural operator methodology in adaptive control and in gain scheduling control for nonlinear PDEs.

show abstract

“…Control-theoretic methods have been explored to learn policies with stability guarantees by constraining policy and system dynamics. Taylor Series IL [27] shows that the induced trajectories of a learner and expert will be close if their derivative difference at expert states is small. However, computing highorder derivatives of the expert policy is difficult without sufficient data.…”

Section: B Imitation Learning With Robustnessmentioning

confidence: 99%

CCIL: Context-conditioned imitation learning for urban driving

Guo,

Jing,

Chen

et al. 2023

Robotics: Science and Systems XIX

View full text Add to dashboard Cite

Imitation learning holds great promise for addressing the complex task of autonomous urban driving, as experienced human drivers can navigate highly challenging scenarios with ease. While behavior cloning is a widely used imitation learning approach in autonomous driving due to its exemption from risky online interactions, it suffers from the covariate shift issue. To address this limitation, we propose a context-conditioned imitation learning approach that employs a policy to map the context state into the ego vehicle's future trajectory, rather than relying on the traditional formulation of both ego and context states to predict the ego action. Additionally, to reduce the implicit ego information in the coordinate system, we design an ego-perturbed goal-oriented coordinate system. The origin of this coordinate system is the ego vehicle's position plus a zero mean Gaussian perturbation, and the x-axis direction points towards its goal position. Our experiments on the real-world large-scale Lyft and nuPlan datasets show that our method significantly outperforms state-of-the-art approaches.

show abstract

TaSIL: Taylor Series Imitation Learning

Cited by 4 publications

References 12 publications

Model Predictive Control via On-Policy Imitation Learning

Model Predictive Control via On-Policy Imitation Learning

Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEs

CCIL: Context-conditioned imitation learning for urban driving

Contact Info

Product

Resources

About