A small-world topology enhances the echo state property and signal propagation in reservoir computing

Kawai, Yuji; Park, Jihoon; Asada, Minoru

doi:10.1016/j.neunet.2019.01.002

Cited by 114 publications

(83 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…On the methodological side, a natural extension of the work in this paper is to analyze the effect of a broader pool of reservoir architectural variants, including e.g. small-world [19], cycles with regular jumps [24] and concentric [1] reservoirs. Moreover, future research could pursue even further the simplification of architectural construction in deep RC models, reducing the impact of randomness in the network initialization in the same vein as the works on minimum complexity ESNs [23,24].…”

Section: Discussionmentioning

confidence: 99%

Reservoir Topology in Deep Echo State Networks

Gallicchio

Micheli

2019

Artificial Neural Networks and Machine Learning – ICANN 2019: Workshop and Special Sessions

View full text Add to dashboard Cite

Deep Echo State Networks (DeepESNs) recently extended the applicability of Reservoir Computing (RC) methods towards the field of deep learning. In this paper we study the impact of constrained reservoir topologies in the architectural design of deep reservoirs, through numerical experiments on several RC benchmarks. The major outcome of our investigation is to show the remarkable effect, in terms of predictive performance gain, achieved by the synergy between a deep reservoir construction and a structured organization of the recurrent units in each layer. Our results also indicate that a particularly advantageous architectural setting is obtained in correspondence of DeepESNs where reservoir units are structured according to a permutation recurrent matrix.

show abstract

Section: Discussionmentioning

confidence: 99%

Reservoir Topology in Deep Echo State Networks

Gallicchio

Micheli

2019

Artificial Neural Networks and Machine Learning – ICANN 2019: Workshop and Special Sessions

View full text Add to dashboard Cite

show abstract

“…We note that the heuristic for the RC practitioners is to assume a random interconnection topology in the reservoir, which usually yields good results. However, regular network topologies also yield optimal results as long as the hyperparameters are optimized, 36,37 as it has been the case here. For the RC method, we only feed a single amplitude value to predict the amplitude of the next pulse.…”

Section: Reservoir Computingmentioning

confidence: 73%

Machine learning algorithms for predicting the amplitude of chaotic laser pulses

Amil

Soriano

Masoller

2019

Chaos: An Interdisciplinary Journal of Nonlinear Science

View full text Add to dashboard Cite

show abstract

“…In graph theory, the clustering coefficient C and the minim characteristic path length L are used to describe the phenomenon of the small-world [10,41]. The reservoir network that is growing naturally consists of 10 3 internal neurons.…”

Section: Small-world Propertymentioning

confidence: 99%

Design of a reservoir for cloud-enabled echo state network with high clustering coefficient

Akrami

Rostami

Khosravi

2020

J Wireless Com Network

View full text Add to dashboard Cite

Reservoir computing (RC) is considered as a suitable alternative for descending gradient methods in recursive neural networks (RNNs) training. The echo state network (ESN) is a platform for RC and nonlinear system simulation in the cloud environment with many external users. In the past researches, the highest eigenvalue of reservoir connection weight (spectral radius) was used to predict reservoir dynamics. Some researchers have illustrated; the characteristics of scale-free and small-world can improve the approximation capability in echo state networks; however, recent studies have shown importance of the infrastructures such as clusters and the stability criteria of these reservoirs as altered. In this research, we suggest a high clustered ESN called HCESN that its internal neurons are interconnected in form of clusters. Each of the clusters contains one backbone and a number of local nodes. We implemented a classical clustering algorithm, called K-means, and three optimization algorithms including genetic algorithm (GA), differential evolution (DE), and particle swarm optimization (PSO) to improve the clustering efficiency of the new reservoir and compared them with each other. For investigating the spectral radius and predictive power of the resulting reservoirs, we also applied them to the laser time series and the Mackey-Glass dynamical system. It is demonstrated that new clustered reservoirs have some specifications of biologic neural systems and complex networks like average short path length, high clustering coefficient, and power-law distribution. The empirical results illustrated that the ESN based on PSO could strikingly enhance echo state property (ESP) and obtains less chaotic time series prediction error compared with other works and the original version of ESN. Therefore, it can approximate nonlinear dynamical systems and predict the chaotic time series.

show abstract

A small-world topology enhances the echo state property and signal propagation in reservoir computing

Cited by 114 publications

References 52 publications

Reservoir Topology in Deep Echo State Networks

Reservoir Topology in Deep Echo State Networks

Machine learning algorithms for predicting the amplitude of chaotic laser pulses

Design of a reservoir for cloud-enabled echo state network with high clustering coefficient

Contact Info

Product

Resources

About