A local Echo State Property through the largest Lyapunov exponent

Wainrib, Gilles; Galtier, Mathieu

doi:10.1016/j.neunet.2015.12.013

Cited by 45 publications

(40 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The trends demonstrate that, in all cases, the decay is consistent with our theoretical predictions. (20), (23), and (28). In all cases, the decay is consistent with the predictions.…”

Section: Hyperbolic Tangentsupporting

confidence: 86%

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Verzelli

Alippi

Livi

2019

Sci Rep

View full text Add to dashboard Cite

Among the various architectures of Recurrent Neural Networks, Echo State Networks (ESNs) emerged due to their simplified and inexpensive training procedure. These networks are known to be sensitive to the setting of hyper-parameters, which critically affect their behavior. Results show that their performance is usually maximized in a narrow region of hyper-parameter space called edge of criticality. Finding such a region requires searching in hyper-parameter space in a sensible way: hyper-parameter configurations marginally outside such a region might yield networks exhibiting fully developed chaos, hence producing unreliable computations. The performance gain due to optimizing hyper-parameters can be studied by considering the memory–nonlinearity trade-off, i.e., the fact that increasing the nonlinear behavior of the network degrades its ability to remember past inputs, and vice-versa. In this paper, we propose a model of ESNs that eliminates critical dependence on hyper-parameters, resulting in networks that provably cannot enter a chaotic regime and, at the same time, denotes nonlinear behavior in phase space characterized by a large memory of past inputs, comparable to the one of linear networks. Our contribution is supported by experiments corroborating our theoretical findings, showing that the proposed model displays dynamics that are rich-enough to approximate many common nonlinear systems used for benchmarking.

show abstract

“…The trends demonstrate that, in all cases, the decay is consistent with our theoretical predictions. (20), (23), and (28). In all cases, the decay is consistent with the predictions.…”

Section: Hyperbolic Tangentsupporting

confidence: 86%

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Verzelli

Alippi

Livi

2019

Sci Rep

View full text Add to dashboard Cite

show abstract

“…These properties come in handy when the goal is to get high exactness of numerical estimates of the LLE, as shown in the work [10]. Recent and different applications using the LLE can be found in [11][12][13][14].…”

Section: Introductionmentioning

confidence: 99%

On the Use of Interval Extensions to Estimate the Largest Lyapunov Exponent from Chaotic Data

Nepomuceno

Martins

Lacerda

et al. 2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

A method to estimate the (positive) largest Lyapunov exponent (LLE) from data using interval extensions is proposed. The method differs from the ones available in the literature in its simplicity since it is only based on three rather simple steps. Firstly, a polynomial NARMAX is used to identify a model from the data under investigation. Secondly, interval extensions, which can be easily extracted from the identified model, are used to calculate the lower bound error. Finally, a simple linear fit to the logarithm of lower bound error is obtained and then the LLE is retrieved from it as the third step. To illustrate the proposed method, the LLE is calculated for the following well-known benchmarks: sine map, Rössler Equations, and Mackey-Glass Equations from identified models given in the literature and also from two identified NARMAX models: a chaotic jerk circuit and the tent map. In the latter, a Gaussian noise has been added to show the robustness of the proposed method.

show abstract

“…We avoid here technical details (see also [49,50]), but this type of result is not valid for any type of RC model: there are , then to scale the matrix:…”

Section: Stability Conditionsmentioning

confidence: 99%

“…On the other direction, to have ESP it is necessary that sr(w r ) ≤ 1, where sr(·) denotes the spectral radius. The gap between these two conditions is unexplored [49]. Note that scaling the reservoir is a computational expensive procedure.…”

Section: Stability Conditionsmentioning

confidence: 99%

Echo State Queueing Networks: A Combination of Reservoir Computing and Random Neural Networks

Basterrech¹,

Rubino²

2017

Prob. Eng. Inf. Sci.

View full text Add to dashboard Cite

This paper deals with two ideas appeared during the last developing phase in Artificial Intelligence: Reservoir Computing (RC) and Random Neural Networks. Both have been very successful in many applications. We propose a new model belonging to the first class, taking the structure of the second for its dynamics. The new model is called Echo State Queuing Network. The paper positions the model in the global Machine Learning area, and provides examples of its use and performances. We show on largely used benchmarks that it is a very accurate tool, and we illustrate how it compares with standard RC models.

show abstract

A local Echo State Property through the largest Lyapunov exponent

Cited by 45 publications

References 42 publications

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

On the Use of Interval Extensions to Estimate the Largest Lyapunov Exponent from Chaotic Data

Echo State Queueing Networks: A Combination of Reservoir Computing and Random Neural Networks

Contact Info

Product

Resources

About