Output Feedback Tube MPC-Guided Data Augmentation for Robust, Efficient Sensorimotor Policy Learning

Abstract

Imitation Learning (IL) has been increasingly employed to generate computationally efficient policies from taskrelevant demonstrations provided by Model Predictive Control (MPC). However, commonly employed IL methods are often data-and computationally-inefficient, as they require a large number of MPC demonstrations, resulting in long training times, and they produce policies with limited robustness to disturbances not experienced during training. In this work, we propose an IL strategy to efficiently compress a computationally expensive MPC into a deep neural network policy that is robust to previously unseen disturbances. By using a robust variant of the MPC, called Robust Tube MPC, and leveraging properties from the controller, we introduce a computationally-efficient data augmentation method that enables a significant reduction of the number of MPC demonstrations and training time required to generate a robust policy. Our approach opens the possibility of zero-shot transfer of a policy trained from a single MPC demonstration collected in a nominal domain, such as a simulation or a robot in a lab/controlled environment, to a new domain with previously unseen bounded model errors/perturbations. Numerical and experimental evaluations performed using linear and nonlinear MPC for agile flight on a multirotor show that our method outperforms strategies commonly employed in IL (such as Dataset-Aggregation (DAgger) and Domain Randomization (DR)) in terms of demonstration-efficiency, training time, and robustness to perturbations unseen during training.

Modeling and LQR Control of Insect Sized Flapping Wing Robot

“…For simplicity, the model we used in this work excludes damping and drag components, thus setting the force and moment vectors [ f a 1 , f a 2 , f a 3 ] T and [L, M, N] T to zero in equations (2)(3)(4)(5)(6)(7).…”

“…The controller used the same Q and R matrices as the hovering maneuver. For comparison, in [7] authors tracked a 5 cm radius circular trajectory with the reported x-y position error of 1.8 cm. Our greater position tracking error is likely due to a much higher flight velocity (25 cm/sec vs a maximum speed of 5.2 cm/s in [7]) and a larger circular trajectory, which would result in larger disturbances by the wire tether.…”

“…However, these systems have not been demonstrated on actual hardware for maneuvers beyond hovering. Additionally, data-driven MPC approaches like Tube-MPC [7] show promise for optimizing under actuator constraints and trajectory tracking for complex maneuvers such as ramps and infinity loops but remain too computationally intensive for implementation on sub-150 mg robots. Microprocessors small enough to be carried onboard, such as the 10 mg, 120 MHz STM32F4 used in the first wireless liftoff of an FIR, the UW Robofly in [1], are capable of floating-point math operations.…”

Tube-NeRF: Efficient Imitation Learning of Visuomotor Policies From MPC via Tube-Guided Data Augmentation and NeRFs

“…The generated policy relies solely on images to obtain information on the robot's horizontal position, which is a challenging task due to (1) its high speed (up to 3.5 m/s), (2) varying altitude, (3) aggressive roll/pitch changes, (4) the sparsity of visual features in our flight space, and (5) the presence of a safety net that moves due to the down-wash of the propellers and that produces semi-transparent visual features above the ground. Contributions: This Letter is an evolution of our previous conference paper [21], where we demonstrated the capabilities of an output-feedback RTMPC-guided DA strategy in simulation. Now, we present a set of algorithmic changes that enabled the first real-world, real-time deployment of the approach.…”

“…Now, we present a set of algorithmic changes that enabled the first real-world, real-time deployment of the approach. Specifically, different from [21], our new algorithm:…”