Large-Scale System Identification Using a Randomized SVD

Abstract

Learning a dynamical system from input/output data is a fundamental task in the control design pipeline. In the partially observed setting there are two components to identification: parameter estimation to learn the Markov parameters, and system realization to obtain a state space model. In both sub-problems it is implicitly assumed that standard numerical algorithms such as the singular value decomposition (SVD) can be easily and reliably computed. When trying to fit a high-dimensional model to data, for example in the cyberphysical system setting, even computing an SVD is intractable. In this work we show that an approximate matrix factorization obtained using randomized methods can replace the standard SVD in the realization algorithm while maintaining the non-asymptotic (in dataset size) performance and robustness guarantees of classical methods. Numerical examples illustrate that for large system models, this is the only method capable of producing a model.

Learning Linear Models Using Distributed Iterative Hessian Sketching

“…Future work will involve benchmarking this second-order method against distributed first-order methods such as accelerated stochastic gradient descent. We are currently integrating this work with our previous results which uses a randomized SVD to produce a system realization (Wang and Anderson, 2021), with the goal of producing end-to-end bounds. Following Pilanci and Wainwright (2016), a "good event" is defined as:…”

“…Then system matrices A, B, C and D can be realized via the Ho-Kalman algorithm (Ho and Kálmán (1966)). In recent work we applied similar ideas based on randomized methods to implement a stochastic version of the Ho-Kalman algorithm suitable for masive-scale problems (Wang and Anderson, 2021). We thus narrow our attention in this work to the task of providing an estimate Ĝ of G.…”

On the Sample Complexity of Stabilizing LTI Systems on a Single Trajectory

“…System Identification. Our work is also related to the system identification literature, which focuses on learning the system parameters of dynamical systems, with early works like Ljung (1999) focusing on asymptotic guarantees, and more recent works such as Fattahi (2021); Oymak and Ozay (2019); Sarkar et al (2019); Simchowitz et al (2018); Wang and Anderson (2021); Xing et al (2021) focusing on finite-time guarantees. Our approach also identifies the system (partially) before constructing a stabilizing controller, but we only identify a part of A rather than the entire A.…”