When Your Robot Breaks: Active Learning During Plant Failure

Abstract

Detecting and adapting to catastrophic failures in robotic systems requires a robot to learn its new dynamics quickly and safely to best accomplish its goals. To address this challenging problem, we propose probabilistically-safe, online learning techniques to infer the altered dynamics of a robot at the moment a failure (e.g., physical damage) occurs. We combine model predictive control and active learning within a chance-constrained optimization framework to safely and efficiently learn the new plant model of the robot. We leverage a neural network for function approximation in learning the latent dynamics of the robot under failure conditions. Our framework generalizes to various damage conditions while being computationally light-weight to advance real-time deployment. We empirically validate within a virtual environment that we can regain control of a severely damaged aircraft in seconds and require only 0.1 seconds to find safe, informationrich trajectories, outperforming state-of-the-art approaches.

Meta-active Learning in Probabilistically-Safe Optimization

“…average. In the high-dimensional domain, Fig 5a depicts the advantage of our method as a function of the number of the number of actions taken to learn the system's dynamical model; we achieve a 49% and 46% improvement over [24] and [25], respectively and a 46% improvement over [27] (Fig. 5b).…”

“…• Maximizing Diversity [25] -This acquisition function selects actions which maximize the difference between previously seen states and actions. • Bayesian Optimization (BaO) [2] -This algorithm was developed for the ADMETS domain (Section 4.1) and is based upon a Gaussian Process model.…”

“…For our ADMETS memory optimization domain, we benchmark against [2,27,26] as those baselines are designed or easily adapted to optimization tasks. For our high-dimensional domain, we likewise benchmark against [2,24,27,25] as those baselines are designed or easily adapted to system identification (i.e., model learning) tasks. We note that we do not compare to MAML [4] or the approach by Nagabandi et al [5], as these algorithms have no notion of safety or active learning.…”

A Meta-Learning-based Trajectory Tracking Framework for UAVs under Degraded Conditions

“…During the online stage, the UAV with an untrained fault was tasked to follow a trajectory with a speed value not included in V. At the beginning of its operation, the UAV adapted the meta-trained network using K = 50 initial data points. The adapted network was used to make predictions and to update the reference trajectory according to (9). During the experiments, we did not apply the trajectory update on the z axis as the faults considered did not cause z deviations.…”

“…In addition to control approaches, machine learning techniques have been also widely used to improve the performance of UAVs under actuator failures or disturbances. In [9], the authors use MPC with active learning to learn the new model of the robot under failure and to provide necessary inputs. Reinforcement Learning (RL) techniques are also utilized to adjust the actuator control commands to compensate for component faults [10], [11].…”