Effects of spectral radius and settling time in the performance of echo state networks

Venayagamoorthy, G.K.; Bashyal, Shristy

doi:10.1016/j.neunet.2009.03.021

Cited by 58 publications

(19 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that, the range of the spectral radius is set to [0.05; 0.9], in steps of 0.05. We can conclude that the system performed well at α = 0.8 and this value is also confirmed by [34].…”

Section: Spectral Radiussupporting

confidence: 79%

“…Jaeger [31] identified some parameters to be appropriately selected for developing an effective ESN: reservoir size, spectral radius of reservoir weight matrix and the scaling of input. For more details Alexandre et al [32], Yuanbiao et al [33], Venayagamoorthy et al [34], Koryakin et al [35] and Rodan et al [36] have also developed some literatures towards the choice of ESN parameters.…”

Section: Echo State Performance and Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Novel Approach Using Echo State Networks for Microscopic Cellular Image Segmentation

2015

View full text Add to dashboard Cite

This paper concentrates on the use of Echo State Networks (ESN), an effective form of reservoir computing, for improving microscopic cellular images segmentation. ESN is a sparsely connected recurrent neural network with most of its weights fixed a priori to randomly chosen values. The only trainable weights are those on links connected to the outputs. The process of segmentation is done with two approaches: the basic form with one reservoir and our approach that corresponds to using multiple reservoirs. Obtained results confirm the benefits of the second approach that outperforms all the state-of-the-art methods considered in this paper for the microscopic image segmentation problem.

show abstract

“…Note that, the range of the spectral radius is set to [0.05; 0.9], in steps of 0.05. We can conclude that the system performed well at α = 0.8 and this value is also confirmed by [34].…”

Section: Spectral Radiussupporting

confidence: 79%

Section: Echo State Performance and Discussionmentioning

confidence: 99%

Novel Approach Using Echo State Networks for Microscopic Cellular Image Segmentation

2015

View full text Add to dashboard Cite

show abstract

“…reservoir has memory or is in the "echo state" (edge of chaos) [46]; however we find no indicative pattern ( Figure 10).…”

Section: Complexitycontrasting

confidence: 54%

Similarity Learning and Generalization with Limited Data: A Reservoir Computing Approach

2018

View full text Add to dashboard Cite

We investigate the ways in which a machine learning architecture known as Reservoir Computing learns concepts such as “similar” and “different” and other relationships between image pairs and generalizes these concepts to previously unseen classes of data. We present two Reservoir Computing architectures, which loosely resemble neural dynamics, and show that a Reservoir Computer (RC) trained to identify relationships between image pairs drawn from a subset of training classes generalizes the learned relationships to substantially different classes unseen during training. We demonstrate our results on the simple MNIST handwritten digit database as well as a database of depth maps of visual scenes in videos taken from a moving camera. We consider image pair relationships such as images from the same class; images from the same class with one image superposed with noise, rotated 90°, blurred, or scaled; images from different classes. We observe that the reservoir acts as a nonlinear filter projecting the input into a higher dimensional space in which the relationships are separable; i.e., the reservoir system state trajectories display different dynamical patterns that reflect the corresponding input pair relationships. Thus, as opposed to training in the entire high-dimensional reservoir space, the RC only needs to learns characteristic features of these dynamical patterns, allowing it to perform well with very few training examples compared with conventional machine learning feed-forward techniques such as deep learning. In generalization tasks, we observe that RCs perform significantly better than state-of-the-art, feed-forward, pair-based architectures such as convolutional and deep Siamese Neural Networks (SNNs). We also show that RCs can not only generalize relationships, but also generalize combinations of relationships, providing robust and effective image pair classification. Our work helps bridge the gap between explainable machine learning with small datasets and biologically inspired analogy-based learning, pointing to new directions in the investigation of learning processes.

show abstract

“…For model selection, in the following experiments, we considered different hyper-parameterizations for reservoir (including the reservoir dimension, the scaling of the matrices collecting input-to-reservoir and internal recurrent reservoir connections weights) and for readout regularization, which have been shown to be relevant for the RC models performance (e.g. [57,13,58,59]).…”

Section: Performance Evaluation Of Treeesnmentioning

confidence: 99%

Tree Echo State Networks

Gallicchio

Micheli

2013

Neurocomputing

View full text Add to dashboard Cite

In this paper we present the Tree Echo State Network (TreeESN) model, generalizing the paradigm of Reservoir Computing to tree structured data. TreeESNs exploit an untrained generalized recursive reservoir, exhibiting extreme efficiency for learning in structured domains. In addition, we highlight through the paper other characteristics of the approach: First, we discuss the Markovian characterization of reservoir dynamics, extended to the case of tree domains, that is implied by the contractive setting of the TreeESN state transition function. Second, we study two types of state mapping functions to map the tree structured state of TreeESN into a fixed-size feature representation for classification or regression tasks. The critical role of the relation between the choice of the state mapping function and the Markovian characterization of the task is analyzed and experimentally investigated on both artificial and real-world tasks. Finally, experimental results on benchmark and real-world tasks show that the TreeESN approach, in spite of its efficiency, can achieve comparable results with state-of-the-art, although more complex, neural and kernel based models for tree structured data

show abstract

Effects of spectral radius and settling time in the performance of echo state networks

Cited by 58 publications

References 2 publications

Novel Approach Using Echo State Networks for Microscopic Cellular Image Segmentation

Novel Approach Using Echo State Networks for Microscopic Cellular Image Segmentation

Similarity Learning and Generalization with Limited Data: A Reservoir Computing Approach

Tree Echo State Networks

Contact Info

Product

Resources

About