Information Processing Capacity of Dynamical Systems

Dambre, Joni; Verstraeten, David; Schrauwen, Benjamin; Massar, Serge

doi:10.1038/srep00514

Cited by 335 publications

(499 citation statements)

References 26 publications

Supporting

Mentioning

434

Contrasting

Order By: Relevance

“…In this spirit, the linear memory capacity [7] has been studied as well as the apparent tradeoff between linear memory and nonlinearity [8]. More recently it has been shown that any dynamical system (under some mild conditions) obeying the fading memory property (which is equivalent to the ESP) essentially has the same amount of computational power with respect to the number of observable variables of the system [9]. Dynamical systems with the ESP do however vary by the precise type of computations they offer, which need to be well-adjusted to the requirements of the application.…”

Section: Echo State Property Revisitedmentioning

confidence: 99%

The spectral radius remains a valid indicator of the Echo state property for large reservoirs

Caluwaerts

wyffels

Dieleman

et al. 2013

The 2013 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

Abstract-In the field of Reservoir Computing, scaling the spectral radius of the weight matrix of a random recurrent neural network to below unity is a commonly used method to ensure the Echo State Property. Recently it has been shown that this condition is too weak. To overcome this problem, othermore involved -sufficient conditions for the Echo State Property have been proposed. In this paper we provide a large-scale experimental verification of the Echo State Property for large recurrent neural networks with zero input and zero bias. Our main conclusion is that the spectral radius method remains a valid indicator of the Echo State Property; the probability that the Echo State Property does not hold, drops for larger networks with spectral radius below unity, which are the ones of practical interest.

show abstract

Section: Echo State Property Revisitedmentioning

confidence: 99%

The spectral radius remains a valid indicator of the Echo state property for large reservoirs

Caluwaerts

wyffels

Dieleman

et al. 2013

The 2013 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

show abstract

“…In principle, any dynamical system which has a high-dimensional phase space is a good candidate for RC [25]. The RC concept offers a highly attractive approach to neuromorphic computation in hardware.…”

Section: The Hardware Reservoirmentioning

confidence: 99%

Advances in photonic reservoir computing

2017

View full text Add to dashboard Cite

Abstract:We review a novel paradigm that has emerged in analogue neuromorphic optical computing. The goal is to implement a reservoir computer in optics, where information is encoded in the intensity and phase of the optical field. Reservoir computing is a bio-inspired approach especially suited for processing time-dependent information. The reservoir's complex and high-dimensional transient response to the input signal is capable of universal computation. The reservoir does not need to be trained, which makes it very well suited for optics. As such, much of the promise of photonic reservoirs lies in their minimal hardware requirements, a tremendous advantage over other hardware-intensive neural network models. We review the two main approaches to optical reservoir computing: networks implemented with multiple discrete optical nodes and the continuous system of a single nonlinear device coupled to delayed feedback.

show abstract

“…A periodic system is characterized by high values for DET (9) and L max (10). In addition, as stressed before, it has low complexity as expressed by low values of ENTR and SWRP.…”

Section: Visualization and Classification Of Reservoir Dynamicsmentioning

confidence: 99%

“…Reservoir computing is a class of state-space models characterized by a fixed state transition structure, the reservoir, and a trainable memoryless readout layer [7]- [9]. A reservoir must be sufficiently complex to capture all salient features of the inputs, behaving as a time-dependent, nonlinear kernel function, which maps the inputs into a higher dimensional space.…”

mentioning

confidence: 99%

Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis

Bianchi

Livi

Alippi

2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Abstract-In this paper, we elaborate over the well-known interpretability issue in echo-state networks (ESNs). The idea is to investigate the dynamics of reservoir neurons with timeseries analysis techniques developed in complex systems research. Notably, we analyze time series of neuron activations with recurrence plots (RPs) and recurrence quantification analysis (RQA), which permit to visualize and characterize highdimensional dynamical systems. We show that this approach is useful in a number of ways. First, the 2-D representation offered by RPs provides a visualization of the high-dimensional reservoir dynamics. Our results suggest that, if the network is stable, reservoir and input generate similar line patterns in the respective RPs. Conversely, as the ESN becomes unstable, the patterns in the RP of the reservoir change. As a second result, we show that an RQA measure, called L max , is highly correlated with the well-established maximal local Lyapunov exponent. This suggests that complexity measures based on RP diagonal lines distribution can quantify network stability. Finally, our analysis shows that all RQA measures fluctuate on the proximity of the so-called edge of stability, where an ESN typically achieves maximum computational capability. We leverage on this property to determine the edge of stability and show that our criterion is more accurate than two well-known counterparts, both based on the Jacobian matrix of the reservoir. Therefore, we claim that RPs and RQA-based analyses are valuable tools to design an ESN, given a specific problem.Index Terms-Dynamics, echo-state network (ESN), recurrence plot (RP), recurrence quantification analysis (RQA), nonlinear time series analysis.

show abstract

Information Processing Capacity of Dynamical Systems

Cited by 335 publications

References 26 publications

The spectral radius remains a valid indicator of the Echo state property for large reservoirs

The spectral radius remains a valid indicator of the Echo state property for large reservoirs

Advances in photonic reservoir computing

Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis

Contact Info

Product

Resources

About