Consistency Hierarchy of Reservoir Computers

Jüngling, Thomas; Lymburn, Thomas; Small, Michael

doi:10.1109/tnnls.2021.3119548

Cited by 6 publications

(10 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, it has been shown that some amount of chaos is not only useful ( 72 ), but also in some cases necessary ( 75 ), for some RNNs to learn complex target functions, because weak chaoticity can expand neural networks’ dynamical repertoire. Finally, although it has often been assumed that chaos inevitably disrupts the consistency of input–output mappings required for computation in some RNNs ( 77 , 78 ), it is now recognized that weakly chaotic systems in general ( 79 – 82 ), and weakly chaotic RNNs in particular ( 72 ), can generate consistent responses to their inputs. However, because input–output consistency generally breaks down in strongly chaotic systems ( 79 , 80 ), it is likewise computationally reasonable for waking cortical electrodynamics to remain only weakly chaotic, in the vicinity of the edge-of-chaos critical point.…”

Section: Discussionmentioning

confidence: 99%

Consciousness is supported by near-critical slow cortical electrodynamics

Toker

Pappas

Lendner

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

103

View full text Add to dashboard Cite

Mounting evidence suggests that during conscious states, the electrodynamics of the cortex are poised near a critical point or phase transition and that this near-critical behavior supports the vast flow of information through cortical networks during conscious states. Here, we empirically identify a mathematically specific critical point near which waking cortical oscillatory dynamics operate, which is known as the edge-of-chaos critical point, or the boundary between stability and chaos. We do so by applying the recently developed modified 0-1 chaos test to electrocorticography (ECoG) and magnetoencephalography (MEG) recordings from the cortices of humans and macaques across normal waking, generalized seizure, anesthesia, and psychedelic states. Our evidence suggests that cortical information processing is disrupted during unconscious states because of a transition of low-frequency cortical electric oscillations away from this critical point; conversely, we show that psychedelics may increase the information richness of cortical activity by tuning low-frequency cortical oscillations closer to this critical point. Finally, we analyze clinical electroencephalography (EEG) recordings from patients with disorders of consciousness (DOC) and show that assessing the proximity of slow cortical oscillatory electrodynamics to the edge-of-chaos critical point may be useful as an index of consciousness in the clinical setting.

show abstract

Section: Discussionmentioning

confidence: 99%

Consciousness is supported by near-critical slow cortical electrodynamics

Toker

Pappas

Lendner

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

103

View full text Add to dashboard Cite

show abstract

“…In addition to assessing delay capacity, we concurrently evaluated two other critical metrics: the covariance rank, as a measure of reservoir diversity, and consistency, as a measure of stability. These properties are vital for optimal reservoir performance 17 , 29 , 31 , 32 . Across all models, the reservoirs that achieve superior performance shown in Fig.…”

Section: Resultsmentioning

confidence: 99%

“…Performance improvements due to such reservoir components and the structures can be substantiated through an analysis of reservoir dynamics 26 . The attributes of reservoir dynamics contributing to time-series prediction can be generally categorised into three areas: memory capacity for input signals 27 , 28 , expressiveness of the output signals 17 , 29 , and consistency between input and output signals known as echo state property 30 – 32 . Concretely, the reservoir must preserve relevant information from the input time series for accurate predictions; therefore, the memory capacity is important 27 .…”

Section: Introductionmentioning

confidence: 99%

Impact of time-history terms on reservoir dynamics and prediction accuracy in echo state networks

Ebato,

Nobukawa,

Sakemi

et al. 2024

Sci Rep

View full text Add to dashboard Cite

The echo state network (ESN) is an excellent machine learning model for processing time-series data. This model, utilising the response of a recurrent neural network, called a reservoir, to input signals, achieves high training efficiency. Introducing time-history terms into the neuron model of the reservoir is known to improve the time-series prediction performance of ESN, yet the reasons for this improvement have not been quantitatively explained in terms of reservoir dynamics characteristics. Therefore, we hypothesised that the performance enhancement brought about by time-history terms could be explained by delay capacity, a recently proposed metric for assessing the memory performance of reservoirs. To test this hypothesis, we conducted comparative experiments using ESN models with time-history terms, namely leaky integrator ESNs (LI-ESN) and chaotic echo state networks (ChESN). The results suggest that compared with ESNs without time-history terms, the reservoir dynamics of LI-ESN and ChESN can maintain diversity and stability while possessing higher delay capacity, leading to their superior performance. Explaining ESN performance through dynamical metrics are crucial for evaluating the numerous ESN architectures recently proposed from a general perspective and for the development of more sophisticated architectures, and this study contributes to such efforts.

show abstract

“…Consistency is a measure of generalized synchronization between nonlinear systems and inputs [36] and can be used as a direct evaluation metric for stability [27,28].…”

Section: Consistencymentioning

confidence: 99%

Impact of Time-History Terms on Reservoir Dynamics and Prediction Accuracy in Echo State Networks

Ebato,

Nobukawa,

Sakemi

et al. 2024

Preprint

View full text Add to dashboard Cite

The echo state network (ESN) is an excellent machine learning model for processing time series data. This model, utilizing the response of a recurrent neural network called a reservoir to input signals, achieves high training efficiency. Introducing time-history terms into the neuron model of the reservoir is known to improve the time series prediction performance of ESN, yet the reasons for this improvement have not been quantitatively explained in terms of reservoir dynamics characteristics. Therefore, we hypothesized that the performance enhancement brought about by time-history terms could be explained by delay capacity, a recently proposed metric for assessing the memory performance of reservoirs. To test this hypothesis, we conducted comparative experiments using ESN models with time-history terms, namely Leaky-Integrator ESNs (LI-ESN) and chaotic echo state networks (ChESN). The results suggest that, compared to ESNs without time-history terms, the reservoir dynamics of LI-ESN and ChESN can maintain diversity and stability while possessing higher delay capacity, leading to their superior performance. Explaining ESN performance through dynamical metrics are crucial for evaluating the numerous ESN architectures recently proposed from a general perspective and for the development of more sophisticated architectures, and this study contributes to such efforts.

show abstract

Consistency Hierarchy of Reservoir Computers

Cited by 6 publications

References 30 publications

Consciousness is supported by near-critical slow cortical electrodynamics

Consciousness is supported by near-critical slow cortical electrodynamics

Impact of time-history terms on reservoir dynamics and prediction accuracy in echo state networks

Impact of Time-History Terms on Reservoir Dynamics and Prediction Accuracy in Echo State Networks

Contact Info

Product

Resources

About