Predictive Modeling with Echo State Networks

Čerňanský, Michal; Tiňo, Peter

doi:10.1007/978-3-540-87536-9_80

Cited by 11 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In comparison with another paper discussing different topologies by Čerňanský and Tiňo [6], our CMA-ES procedure reached Even though the results presented in [6] are slightly surpassed by our method, the authors have reached impressive performance simply by dividing the reservoir matrix by its largest eigenvalue. This scaling technique transforms the reservoir to be close to the edge of chaos dynamics.…”

Section: Resultsmentioning

confidence: 58%

Hyperparameter tuning in echo state networks

Matzner

2022

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Echo State Networks represent a type of recurrent neural network with a large randomly generated reservoir and a small number of readout connections trained via linear regression. The most common topology of the reservoir is a fully connected network of up to thousands of neurons. Over the years, researchers have introduced a variety of alternative reservoir topologies, such as a circular network or a linear path of connections. When comparing the performance of different topologies or other architectural changes, it is necessary to tune the hyperparameters for each of the topologies separately since their properties may significantly differ. The hyperparameter tuning is usually carried out manually by selecting the best performing set of parameters from a sparse grid of predefined combinations. Unfortunately, this approach may lead to underperforming configurations, especially for sensitive topologies. We propose an alternative approach of hyperparameter tuning based on the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Using this approach, we have improved multiple topology comparison results by orders of magnitude suggesting that topology alone does not play as important role as properly tuned hyperparameters.

show abstract

Section: Resultsmentioning

confidence: 58%

Hyperparameter tuning in echo state networks

Matzner

2022

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

show abstract

“…To provide a quantitative support to our analysis, we experimentally assess the L-deepESN model on a variety of progressively more involving versions of the Multiple Superimposed Oscillator (MSO) task [22,23]. Note that the class of MSO tasks is of particular interest for the aims of this paper, especially in light of previous literature results that pointed out the relevant need for multiple time-scales processing ability [13,20,23] as well as the potentiality of linear models in achieving excellent predictive results in base settings of the problem [2]. Another example of application of linear RNNs is in [17].…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Temporal Representation in Linear Reservoir Computing

Gallicchio

Micheli

Pedrelli

2018

Neural Advances in Processing Nonlinear Dynamic Signals

View full text Add to dashboard Cite

Recently, studies on deep Reservoir Computing (RC) highlighted the role of layering in deep recurrent neural networks (RNNs). In this paper, the use of linear recurrent units allows us to bring more evidence on the intrinsic hierarchical temporal representation in deep RNNs through frequency analysis applied to the state signals. The potentiality of our approach is assessed on the class of Multiple Superimposed Oscillator tasks. Furthermore, our investigation provides useful insights to open a discussion on the main aspects that characterize the deep learning framework in the temporal domain.

show abstract

“…ESN is a reservoir computing (RC) method that provides a supervised learning architecture for recurrent neural networks (RNNs). It is biologically more plausible than other forms of artificial neural networks (ANN) [3] such as multilayer perceptron (MLP), and its training process is conceptually simple [4]. Furthermore, to the best of our knowledge, there is no approach in the literature that considers the use of ESNs to perform radio signal strength predictions.…”

Section: Introductionmentioning

confidence: 99%