Growing Echo-State Network With Multiple Subreservoirs

Qiao, Junfei; Li, Fanjun; Han, Honggui; Li, Wenjing

doi:10.1109/tnnls.2016.2514275

Cited by 164 publications

(42 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A scale-free highly-clustered ESN was proposed in [20], which used a small world network and scale-free network topology to create a reservoir. A growing ESN was proposed in the approach of [21], which could set the size and topology of the reservoir automatically. The simulation results showed that the growing ESN is better than the original ESN (with fixed size and topology) in predictive performance and learning speed; (b).…”

Section: Introductionmentioning

confidence: 99%

An echo state network architecture based on quantum logic gate and its optimization

et al. 2020

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

An echo state network architecture based on quantum logic gate and its optimization

et al. 2020

View full text Add to dashboard Cite

show abstract

“…To guarantee ESP without the reservoir weight scale, a reservoir production method has been proposed using the eigenvalue decomposition, and the convergence of the algorithm is also theoretically guaranteed. However, eigenvalues are still generated randomly [9].…”

Section: Introductionmentioning

confidence: 99%

Design of a reservoir for cloud-enabled echo state network with high clustering coefficient

Akrami

Rostami

Khosravi

2020

J Wireless Com Network

View full text Add to dashboard Cite

Reservoir computing (RC) is considered as a suitable alternative for descending gradient methods in recursive neural networks (RNNs) training. The echo state network (ESN) is a platform for RC and nonlinear system simulation in the cloud environment with many external users. In the past researches, the highest eigenvalue of reservoir connection weight (spectral radius) was used to predict reservoir dynamics. Some researchers have illustrated; the characteristics of scale-free and small-world can improve the approximation capability in echo state networks; however, recent studies have shown importance of the infrastructures such as clusters and the stability criteria of these reservoirs as altered. In this research, we suggest a high clustered ESN called HCESN that its internal neurons are interconnected in form of clusters. Each of the clusters contains one backbone and a number of local nodes. We implemented a classical clustering algorithm, called K-means, and three optimization algorithms including genetic algorithm (GA), differential evolution (DE), and particle swarm optimization (PSO) to improve the clustering efficiency of the new reservoir and compared them with each other. For investigating the spectral radius and predictive power of the resulting reservoirs, we also applied them to the laser time series and the Mackey-Glass dynamical system. It is demonstrated that new clustered reservoirs have some specifications of biologic neural systems and complex networks like average short path length, high clustering coefficient, and power-law distribution. The empirical results illustrated that the ESN based on PSO could strikingly enhance echo state property (ESP) and obtains less chaotic time series prediction error compared with other works and the original version of ESN. Therefore, it can approximate nonlinear dynamical systems and predict the chaotic time series.

show abstract

“…The input-to-hidden and hidden-to-hidden (recurrent) weights are randomly initialized and fixed during the learning stage. By learning the state dynamics generated from the untrained recurrent hidden layer, ESNs can avoid the laborious process of gradient-descent RNN training, yet achieve excellent performance in time series prediction (Chatzis & Demiris, 2011;Jaeger & Haas, 2004;Li, Han, & Wang, 2012;Lukoševičius & Jaeger, 2009;Qiao, Li, Han, & Li, 2017;Shi & Han, 2007), speech recognition (TSC) tasks has not been fully explored. In recent years, some researchers have applied ESNs to TSC tasks.…”

Section: Introductionmentioning

confidence: 99%

Time series classification with Echo Memory Networks

Zhuang

Shen

et al. 2019

Neural Networks

View full text Add to dashboard Cite

a b s t r a c tEcho state networks (ESNs) are randomly connected recurrent neural networks (RNNs) that can be used as a temporal kernel for modeling time series data, and have been successfully applied on time series prediction tasks. Recently, ESNs have been applied to time series classification (TSC) tasks. However, previous ESN-based classifiers involve either training the model by predicting the next item of a sequence, or predicting the class label at each time step. The former is essentially a predictive model adapted from time series prediction work, rather than a model designed specifically for the classification task. The latter approach only considers local patterns at each time step and then averages over the classifications. Hence, rather than selecting the most discriminating sections of the time series, this approach will incorporate non-discriminative information into the classification, reducing accuracy. In this paper, we propose a novel end-to-end framework called the Echo Memory Network (EMN) in which the time series dynamics and multi-scale discriminative features are efficiently learned from an unrolled echo memory using multi-scale convolution and max-over-time pooling. First, the time series data are projected into the high dimensional nonlinear space of the reservoir and the echo states are collected into the echo memory matrix, followed by a single multi-scale convolutional layer to extract multi-scale features from the echo memory matrix. Max-over-time pooling is used to maintain temporal invariance and select the most important local patterns. Finally, a fully-connected hidden layer feeds into a softmax layer for classification. This architecture is applied to both time series classification and human action recognition datasets. For the human action recognition datasets, we divide the action data into five different components of the human body, and propose two spatial information fusion strategies to integrate the spatial information over them. With one training-free recurrent layer and only one layer of convolution, the EMN is a very efficient end-to-end model, and ranks first in overall classification ability on 55 TSC benchmark datasets and four 3D skeleton-based human action recognition tasks.

show abstract

Growing Echo-State Network With Multiple Subreservoirs

Cited by 164 publications

References 31 publications

An echo state network architecture based on quantum logic gate and its optimization

An echo state network architecture based on quantum logic gate and its optimization

Design of a reservoir for cloud-enabled echo state network with high clustering coefficient

Time series classification with Echo Memory Networks

Contact Info

Product

Resources

About