Divergence-based characterization of fundamental limitations of adaptive dynamical systems

Abstract

Adaptive dynamical systems arise in a multitude of contexts, e.g., optimization, control, communications, signal processing, and machine learning. A precise characterization of their fundamental limitations is therefore of paramount importance. In this paper, we consider the general problem of adaptively controlling and/or identifying a stochastic dynamical system, where our a priori knowledge allows us to place the system in a subset of a metric space (the uncertainty set). We present an information-theoretic meta-theorem that captures the trade-off between the metric complexity (or richness) of the uncertainty set, the amount of information acquired online in the process of controlling and observing the system, and the residual uncertainty remaining after the observations have been collected. Following the approach of Zames, we quantify a priori information by the Kolmogorov (metric) entropy of the uncertainty set, while the information acquired online is expressed as a sum of information divergences. The general theory is used to derive new minimax lower bounds on the metric identification error, as well as to give a simple derivation of the minimum time needed to stabilize an uncertain stochastic linear system.

Learning-Based Attacks in Cyber-Physical Systems

“…On the other hand, by looking at the denominator, the bound becomes tighter as R increases. This is consistent with the observation of Zames [58] that system identification becomes harder as the uncertainty about the open-loop gain of the plant increases. In our case, a larger uncertainty interval R corresponds to a poorer estimation of A by the attacker, which leads, in turn, to a decrease in the achievable deception probability.…”

“…whenever √ δβ ≤ R. Finally, (14) follows the arguments of [58] and is proven, for completeness, in App. D-B [52].…”

Sample Complexity Lower Bounds for Linear System Identification

“…Finally, we can mention [17] where the author derives lower bounds of the time required to achieve a particular objective in adaptive control. Specifically, the time required to achieve, in LQG problems, a regret no greater than a fixed fraction of time (linear regret) is investigated.…”

“…These sample complexity lower bounds are typically derived in a minimax framework (they characterize the minimal number of observations required for the worst system), and are restricted to specific subclasses of systems [9]. These limitations are shared by earlier studies on the sample complexity of LTI system identification [16], [17]. Refer to Section II for a more exhaustive survey.…”

On a Phase Transition of Regret in Linear Quadratic Control: The Memoryless Case

“…It would also be interesting to relax or replace the assumption of (1/2)-unbiasedness in Theorem V.2 for the full Ω( √ T )-lower bound. Another potential direction is to take a more directly information-theoretic route toward lower bounds as in [24] and as is traditional for bandits [20]. This was recently done for system identification in [25].…”