Nonparametric System identification of Stochastic Switched Linear Systems

Sarkar, Tanmoy; Rakhlin, Alexander; Dahleh, Munther A.

doi:10.1109/cdc40024.2019.9030173

Cited by 19 publications

(22 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These guarantees focus more heavily on error sensitivity and stability; they only permit dynamical recovery up to error that scales linearly in system noise, and thus guarantee only (vacuous) linear-in-horizon regret. More recent work has studied identification (but not online control) of an important LTV class called switching systems [31,35].…”

Section: Related Workmentioning

confidence: 99%

Online Control of Unknown Time-Varying Dynamical Systems

Minasyan¹,

Gradu²,

Simchowitz³

et al. 2022

Preprint

View full text Add to dashboard Cite

We study online control of time-varying linear systems with unknown dynamics in the nonstochastic control model. At a high level, we demonstrate that this setting is qualitatively harder than that of either unknown time-invariant or known time-varying dynamics, and complement our negative results with algorithmic upper bounds in regimes where sublinear regret is possible. More specifically, we study regret bounds with respect to common classes of policies: Disturbance Action (SLS), Disturbance Response (Youla), and linear feedback policies. While these three classes are essentially equivalent for LTI systems, we demonstrate that these equivalences break down for time-varying systems.We prove a lower bound that no algorithm can obtain sublinear regret with respect to the first two classes unless a certain measure of system variability also scales sublinearly in the horizon. Furthermore, we show that offline planning over the state linear feedback policies is NP-hard, suggesting hardness of the online learning problem.On the positive side, we give an efficient algorithm that attains a sublinear regret bound against the class of Disturbance Response policies up to the aforementioned system variability term. In fact, our algorithm enjoys sublinear adaptive regret bounds, which is a strictly stronger metric than standard regret and is more appropriate for time-varying systems. We sketch extensions to Disturbance Action policies and partial observation, and propose an inefficient algorithm for regret against linear state feedback policies.

show abstract

Section: Related Workmentioning

confidence: 99%

Online Control of Unknown Time-Varying Dynamical Systems

Minasyan¹,

Gradu²,

Simchowitz³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Instead, we discuss recent studies of system identification that develop finite time statistical guarantees. Most recent theoretical guarantees of system identification apply to linear systems under various sets of assumptions [11,13,15,16,20,21,22,36,41,42,43,47,48,53,54,55]. Notably, Simchowitz et al [47] derived sharp rates for the non-adaptive estimation of marginally stable systems.…”

Section: Related Workmentioning

confidence: 99%

Active Learning for Nonlinear System Identification with Guarantees

Mania¹,

Jordan²,

Recht³

2020

Preprint

View full text Add to dashboard Cite

While the identification of nonlinear dynamical systems is a fundamental building block of model-based reinforcement learning and feedback control, its sample complexity is only understood for systems that either have discrete states and actions or for systems that can be identified from data generated by i.i.d. random inputs. Nonetheless, many interesting dynamical systems have continuous states and actions and can only be identified through a judicious choice of inputs. Motivated by practical settings, we study a class of nonlinear dynamical systems whose state transitions depend linearly on a known feature embedding of state-action pairs. To estimate such systems in finite time identification methods must explore all directions in feature space. We propose an active learning approach that achieves this by repeating three steps: trajectory planning, trajectory tracking, and re-estimation of the system from all available data. We show that our method estimates nonlinear dynamical systems at a parametric rate, similar to the statistical rate of standard linear regression.

show abstract

“…High-probability light tail bounds are therefore not applicable without very strong assumptions on the joint spectral radius of different modes (cf. [56]). Perhaps more surprisingly, there are examples of MJS with all modes individually stable, however due to switching, the system exhibits an unstable behavior on average, and the MJS is not mean-square stable (see Example 3.17 of [14]).…”

Section: Introductionmentioning

confidence: 99%

Identification and Adaptive Control of Markov Jump Systems: Sample Complexity and Regret Bounds

Sattar¹,

Du²,

Tarzanagh³

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning how to effectively control unknown dynamical systems is crucial for intelligent autonomous systems. This task becomes a significant challenge when the underlying dynamics are changing with time. Motivated by this challenge, this paper considers the problem of controlling an unknown Markov jump linear system (MJS) to optimize a quadratic objective. By taking a model-based perspective, we consider identification-based adaptive control for MJSs. We first provide a system identification algorithm for MJS to learn the dynamics in each mode as well as the Markov transition matrix, underlying the evolution of the mode switches, from a single trajectory of the system states, inputs, and modes. Through mixing-time arguments, sample complexity of this algorithm is shown to be O(1 √ T ). We then propose an adaptive control scheme that performs system identification together with certainty equivalent control to adapt the controllers in an episodic fashion. Combining our sample complexity results with recent perturbation results for certainty equivalent control, we prove that when the episode lengths are appropriately chosen, the proposed adaptive control scheme achieves O( √ T ) regret, which can be improved to O(polylog(T )) with partial knowledge of the system. Our proof strategy introduces innovations to handle Markovian jumps and a weaker notion of stability common in MJSs. Our analysis provides insights into system theoretic quantities that affect learning accuracy and control performance. Numerical simulations are presented to further reinforce these insights.

show abstract

Nonparametric System identification of Stochastic Switched Linear Systems

Cited by 19 publications

References 28 publications

Online Control of Unknown Time-Varying Dynamical Systems

Online Control of Unknown Time-Varying Dynamical Systems

Active Learning for Nonlinear System Identification with Guarantees

Identification and Adaptive Control of Markov Jump Systems: Sample Complexity and Regret Bounds

Contact Info

Product

Resources

About