Model-free control of dynamical systems with deep reservoir computing

Abstract

We propose and demonstrate a nonlinear control method that can be applied to unknown, complex systems where the controller is based on a type of artificial neural network known as a reservoir computer. In contrast to many modern neural-network-based control techniques, which are robust to system uncertainties but require a model nonetheless, our technique requires no prior knowledge of the system and is thus model-free. Further, our approach does not require an initial system identification step, resulting in a relatively simple and efficient learning process. Reservoir computers are well-suited to the control problem because they require small training data sets and remarkably low training times. By iteratively training and adding layers of reservoir computers to the controller, a precise and efficient control law is identified quickly. With examples on both numerical and high-speed experimental systems, we demonstrate that our approach is capable of controlling highly complex dynamical systems that display deterministic chaos to nontrivial target trajectories.

Next generation reservoir computing

“…For example, the NG-RC can be used to create a digital twin for dynamical systems 33 using only observed data or by combining approximate models with observations for data assimilation 34,35 . It can also be used for nonlinear control of dynamical systems 36 , which can be quickly adjusted to account for changes in the system, or for speeding up the simulation of turbulence 37 . Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.…”

Physical reservoir computing using finitely-sampled quantum systems

“…The solid blue curve is the exact complex-P result for Λ/γ = 0.005, while blue dots are obtained by evolving the TEOMs, Eqs. (10), (11a), (11b), to their steady state numerically. For small N , the response follows that of a linear oscillator (dashed black line), but becomes nonlinear with increasing N .…”

“…Reservoir computing is a machine-learning paradigm that emphasizes learning efficiency [1-4]: a linear combination of the accessible degrees of freedom of a physical 'reservoir' is the only quantity trained for a given task. Crucially, foregoing optimization of internal parameters does not necessarily hinder the computational capacity of reservoir computers, and has in recent years enabled efficient approaches to forecasting [5][6][7], inference [8,9], control [10], and similar resource-intensive signal processing tasks [11][12][13][14][15]. By relaxing control over microscopic parameters of the learning machine, reservoir computing embraces the fundamental notion of computation with very general dynamical systems: physical or artificial dynamical systems stimulated by incoming time-dependent signals u(t) perform computation by transforming an input stream into an output stream, effectively computing a function F{u(t)} [see Fig.…”

Model-free tracking control of complex dynamical trajectories with machine learning

“…Reservoir computing and its variant echo state Gaussian process [57] were also used in model predictive control of unknown nonlinear dynamical systems [58,59], which served as replacements of the traditional recurrent neural-network models with low computational cost. More recently, deep reservoir networks were proposed for controlling chaotic systems [60].…”