Control-Oriented Learning on the Fly

Abstract

This paper focuses on developing a strategy for control of systems whose dynamics are almost entirely unknown.This situation arises naturally in a scenario where a system undergoes a critical failure. In that case, it is imperative to retain the ability to satisfy basic control objectives in order to avert an imminent catastrophe. A prime example of such an objective is the reach-avoid problem, where a system needs to move to a certain state in a constrained state space. To deal with limitations on our knowledge of system dynamics, we develop a theory of myopic control.The primary goal of myopic control is to, at any given time, optimize the current direction of the system trajectory, given solely the information obtained about the system until that time. We propose an algorithm that uses small perturbations in the control effort to learn local dynamics while simultaneously ensuring that the system moves in a direction that appears to be nearly optimal, and provide hard bounds for its suboptimality. We additionally verify the usefulness of the algorithm on a simulation of a damaged aircraft seeking to avoid a crash, as well as on an example of a Van der Pol oscillator.

Reachability of Nonlinear Systems with Unknown Dynamics

“…The case of full actuation in [1] corresponds the case where R = I n in Assumption 1. Motivated by the online learning technique introduced in [7], we make the following assumption about the knowledge regarding the system dynamics.…”

“…The primary contribution of this paper is to provide a meaningful underapproximation of the GRS of a control-affine system. We assume the only available information at the time of computation consists of (i) local dynamics at a single point, which can be obtained with an arbitrarily small error from applying test control inputs over a short period of time [7], and (ii) Lipschitz bounds on the rate of change of the system's dynamics provided by prior knowledge of the system design and physical laws. The reachable set of an unknown system is impossible to compute.…”

“…The reachable set of an unknown system is impossible to compute. Prior work determining the reachable set often focused on overapproximations [7], [8]. Without discussing reachable sets, the work in [7], operating under similar assumptions as our paper, focused on optimistic reachability, i.e., attempting to reach a particular objective while there exists any chance of reaching it.…”

“…Prior work determining the reachable set often focused on overapproximations [7], [8]. Without discussing reachable sets, the work in [7], operating under similar assumptions as our paper, focused on optimistic reachability, i.e., attempting to reach a particular objective while there exists any chance of reaching it. Conversely, this paper computes states that are guaranteed to be reachable using admissible control signals.…”

“…Apart from [7], work on reachability under uncertainty considered computation of reachable sets with dynamics generated by a finite number of uncertain parameters [9], [10] or having bounded disturbances [11], [12]. Work in adaptive and robust control [8], [13], [14] assumes more knowledge on the magnitude or structure of the system dynamics, and classical data-driven learning methods [2], [15], [16] collect data through repeated system runs.…”