Transfer learning of chaotic systems

Abstract

Can a neural network trained by the time series of system A be used to predict the evolution of system B? This problem, knowing as transfer learning in a broad sense, is of great importance in machine learning and data mining yet has not been addressed for chaotic systems. Here, we investigate transfer learning of chaotic systems from the perspective of synchronization-based state inference, in which a reservoir computer trained by chaotic system A is used to infer the unmeasured variables of chaotic system B, while A is different from B in either parameter or dynamics. It is found that if systems A and B are different in parameter, the reservoir computer can be well synchronized to system B. However, if systems A and B are different in dynamics, the reservoir computer fails to synchronize with system B in general. Knowledge transfer along a chain of coupled reservoir computers is also studied, and it is found that, although the reservoir computers are trained by different systems, the unmeasured variables of the driving system can be successfully inferred by the remote reservoir computer. Finally, by an experiment of chaotic pendulum, we demonstrate that the knowledge learned from the modeling system can be transferred and used to predict the evolution of the experimental system.

Learning the dynamics of coupled oscillators from transients

“…Model-free prediction of chaotic systems by the technique of reservoir computer (RC) in machine learning has received considerable attention in recent years [24][25][26][27][28][29][30][31][32][33][34][35][36][37]. From the perspective of dynamical systems, RC can be regarded as a complex network of coupled nonlinear units which, driven by the input signals, generates the outputs through a readout function [38,39].…”

“…For instance, by incorporating a parameter-control channel into RC, it is shown that the machine trained by the time series of several states in the oscillatory regime of a dynamical system is able to predict not only the critical point for system collapse, but also the averaged lifetime of the transients in the postcritical regime [32]; training the machine by the time series of coupled oscillators at several states in the desynchronization regime, the machine is able to predict accurately the critical coupling for synchronization [33]. It is noted that in predicting chaotic systems by the technique of RC, the training data are all measured from the asymptotic dynamics that the systems are finally developed to, while the transient behaviors preceding the asymptotic dynamics is normally discarded [24][25][26][27][28][29][30][31][32][33][34][35][36][37].…”

Learning Hamiltonian dynamics by reservoir computer

“…Besides predicting chaos evolutions, RC has also been exploited to address other long-standing questions in nonlinear science, saying, for example, reconstructing chaotic attractors and calculating Lyapunov exponents [17], synchronizing chaotic oscillators [8,18], predicting system collapses [19], reconstructing synchronization transition paths [20], transferring knowledge between different systems [21,22], to name just a few. These studies, while demonstrating the power of RC in solving different nonlinear questions, also give insights on the working mechanisms of RC.…”

“…Exploiting this ability, model-free techniques have been proposed in recent years to predict the bifurcations in nonlinear dynamical systems, e.g., reproducing the bifurcation diagram of classical chaotic systems [13,14], anticipating the critical points of system collapse [19], predicting the critical coupling for synchronization [20], etc. Another property revealed recently in exploiting RC is that knowledge can be transferred between different dynamical systems, namely the ability of transfer learning [21,22]. Specifically, it is shown that the RC trained by the time series of system A can be used to infer the properties of system B, with the motions of A and B significantly different from each other.…”

Criticality in Reservoir Computer of Coupled Phase Oscillators

“…(See Supplementary Material for more details.) These findings are reminiscent of the transfer learning of chaotic systems [8,12,52,53], where it is shown that the RC trained by the time series of a periodic oscillator can be used to infer the statistical properties of a chaotic oscillator with the same type of dynamics but different bifurcation parameters. The fact that knowledge can be transferred between different systems suggests that it is the intrinsic dynamics that the machine leans from the data, instead of the mathematical expressions describing the trajectories.…”