Consistency in echo-state networks

Abstract

Consistency is an extension to generalized synchronization which quantifies the degree of functional dependency of a driven nonlinear system to its input. We apply this concept to echo-state networks, which are an artificial-neural network version of reservoir computing. Through a replica test we measure the consistency levels of the high-dimensional response, yielding a comprehensive portrait of the echo-state property.When a nonlinear dynamical system is externally modulated by an information-carrying signal, its erratic response hides an intricate property: Consistency. It is difficult to estimate from time series whether or not the variability in the output is entirely determined by the driving signal. For autonomous chaotic systems it is well-known that their inherent instability gives rise to a certain level of unpredictability. For a driven system, this means that a part of the variability of its output does not depend on the drive. Consistency quantifies the degree of this dependency through a replica test. The nonlinear system is repeatedly driven by the same signal, and the corresponding responses are compared. We apply this concept to echo-state networks, a class of artificial neural networks with a fixed random internal connectivity. Such networks have been successfully utilized for sequential processing tasks like nonlinear time series prediction and spoken digit recognition. Studying the consistency property allows for a more comprehensive understanding of the dynamical response and for tailoring the network systematically towards enhanced functionality and a wider range of applications.

Performance boost of time-delay reservoir computing by non-resonant clock cycle

“…have memory. References [10] and [11] study how a reservoir can store information about past inputs. Second, the system should contain a non-linearity to allow for non-trivial data processing.…”

The reservoir’s perspective on generalized synchronization

“…wheres(t) ands ′ (t) denote the time series normalized to zero mean and unit variance. This definition of the consistency correlation γ 2 gives credit to the fact that its square-root γ poses a fundamental limit to any approximation of the response from only the input signal 37,60 .…”

“…It is now widely accepted that several hard and soft criteria need to be satisfied by the reservoir to perform its function. These include fading memory 33,34 and consistency [35][36][37] , amongst others. A generic practical guide to setting up ESNs is given in Ref.…”

Analysis of Wide and Deep Echo State Networks for Multiscale Spatiotemporal Time Series Forecasting

“…Various metrics have been proposed to understand reservoir performance [15,16,23,28,37]. In this work, we quantify reservoir goodness by measuring the stability of reservoir dynamics as well as evaluating the forecasting proficiency of networks trained on synthetic and real-world tasks.…”