A Systematic Review of Echo State Networks From Design to Application

Sun, C. P.; Song, Moxian; Cai, Derun; Zhang, Baofeng; Hong, Shenda; Li, Hongyan

doi:10.1109/tai.2022.3225780

Cited by 18 publications

(10 citation statements)

References 108 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The unpredictability and complexity of solar activity result in strong nonlinearities in sunspot sequences, making them suitable for testing network models' ability to solve real-world time-series prediction problems. We use a 13-month open-source monthly unsmoothed sunspot series provided by the World Data Centre SILSO, [40] collected from January 1749 to March 2021, containing 3267 sunspot sampling points. In our experiment, the first 2200 sequence sampling points serve as the network training set, 2200-2400 sequence sampling points as the validation set, and 2400-3000 sequence sampling points as the test set, performing one-step ahead direct prediction.…”

Section: Dataset Introductionmentioning

confidence: 99%

Exploring reservoir computing: Implementation via double stochastic nanowire networks

Tang 唐,

Xia 夏,

Li 李

et al. 2024

Chinese Phys. B

View full text Add to dashboard Cite

Neuromorphic computing, inspired by the human brain, uses memristor devices for complex tasks. Recent studies show that self-organizing random nanowires can implement neuromorphic information processing, enabling data analysis. This paper presents a model based on these nanowire networks, with an improved conductance variation profile. We suggest using these networks for temporal information processing via a reservoir computing scheme and propose an efficient data encoding method using voltage pulses. The nanowire network layer generates dynamic behaviors for pulse voltages, allowing time series prediction analysis. Our experiment uses a double stochastic nanowire network architecture (DS-nanowire network) for processing multiple input signals, outperforming traditional reservoir computing in terms of fewer nodes, enriched dynamics, and improved prediction accuracy. Experimental results confirm the high accuracy of this architecture on multiple real-time series datasets, making neuromorphic nanowire networks promising for physical implementation of reservoir computing.

show abstract

Section: Dataset Introductionmentioning

confidence: 99%

Exploring reservoir computing: Implementation via double stochastic nanowire networks

Tang 唐,

Xia 夏,

Li 李

et al. 2024

Chinese Phys. B

View full text Add to dashboard Cite

show abstract

“…Furthermore, as only the output layer is trained in RC, it is unclear whether it can achieve comparable prediction accuracy for real-world applications to other state-of-the approaches, given the same energy consumption [14]. To improve the computational efficiency of RC, various RC architectures and methods have been proposed [15]. Recent proposals include structures…”

mentioning

confidence: 99%

Learning Reservoir Dynamics with Temporal Self-Modulation

Sakemi

Nobukawa

Matsuki

et al. 2023

Preprint

View full text Add to dashboard Cite

Reservoir computing (RC) can efficiently process time-series data by transferring the input signal to randomly connected recurrent neural networks (RNNs), which are referred to as a reservoir. The high-dimensional representation of time-series data in the reservoir significantly simplifies subsequent learning tasks. Although this simple architecture allows fast learning and facile physical implementation, the learning performance is inferior to that of other state-of-the-art RNN models. In this study, to improve the learning ability of RC, we propose self-modulated RC (SM-RC) that extends RC by adding a self-modulation mechanism. We demonstrated that SM-RC can perform attention tasks where input information is retained or discarded depending on the input signal. We also found that a chaotic state emerged as a result of learning in SM-RC. Furthermore, SM-RC significantly outperformed RC in NARMA and Lorentz model tasks. Because the SM-RC architecture only requires two additional gates, it is physically implementable as RC, thereby providing a new direction for realizing edge AI.

show abstract

“…ESN model is suitable for non-linear approximation problems such as: Identification Systems (SCHWEDERSKY; FLESCH; DANGUI, 2022), Time Series Prediction (ZHENG et al, 2020), Pattern Recognition (JAMSHIDI; DANESHFAR, 2022), Modeling Neural Plasticity for Classification and Regression (SUN et al, 2022), among others. The reservoir of ESN model is used as a processing layer and is not modified during its training phase.…”

Section: Introductionmentioning

confidence: 99%

“…For a good performance, this reservoir must satisfy a condition about its dynamics state (echo property, see 2.1.1. This property states that the dynamic state of the reservoir is influenced by the spectral radius, the highest eigenvalue of a matrix, which is a parameter that has a high impact over the performance model and the capacity of good estimations (SUN et al, 2022). Although the ESN model inherits the main benefit of RC techniques (simple training phase), they have been criticized because the configuration about connections between internal units are generated randomly, ie.…”

Section: Introductionmentioning

confidence: 99%

“…According to (JAEGER, 2002), the more complex the task to be performed, the more complex the reservoir structure is required. In consequence, when the tasks turn more difficult, the stability of the trained dynamics becomes a critical issue for the ESN training phase (SUN et al, 2022) and a random configurations into internal units of the reservoir becomes inadequate.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Modified Echo State Network Model Using Non-Random Topology

Arroyo

View full text Add to dashboard Cite

A computação de reservatório é um tipo de redes neurais recorrentes adequadas para o processamento de dados temporais/sequenciais. Um dos modelos representativos da computação de reservatório é a rede de estado de eco (no inglês Echo State Network, ESN), que mapeia os padrões de entrada-saída através de uma projeção não linear de alta dimensão, denominada reservatório. No modelo clássico de ESN, o reservatório é composto por um grande número de neurónios com ligações aleatórias, formando uma rede complexa aleatória do tipo Erdös-Rényi. Desta forma, o reservatório mapeia os dados de entrada num espaço de dimensão superior para resolver o problema da inseparabilidade linear, aplicando um algoritmo de regressão linear simples na fase de treino. Inspirados na estrutura modular de um cérebro humano, propomos novas ESNs utilizando topologia não aleatória como reservatório por modelos de redes complexas e modelos gerados via agrupamento de dados. A topologia de conetividade baseada em redes complexas no reservatório são: redes aleatórias, redes livre de escala e redes de mundo pequeno. Para gerar os reservatórios com clusters, propomos utilizar os algoritmos clássicos de agrupamento de dados: K-means, Partitioning Around Medoids, e algoritmo de Ward para simular estruturas de comunidades. Também geramos as redes livre de escala e de pequeno mundo agrupadas como reservatórios. A principal hipótese desta abordagem é que as estruturas de rede não aleatórias, especialmente as redes com clusters como reservatórios, podem captar melhor a informação de diferentes classes dos dados de treino, de tal forma que um grupo de comunidades da rede pode servir para codificar uma determinada classe de dados. Simulações numéricas em aplicações de previsão de sinais mostram que os modelos propostos apresentam uma melhoria no seu desempenho e um custo computacional mais baixo em comparação com as ESN clássicas. A utilidade dos nossos métodos baseados em ESNs foi demonstrada considerando a classificação de tecidos em imagens histopatológicas de lâminas inteiras (WSI) coradas com hematoxilina e eosina (H&E). Os modelos propostos para classificar componentes de tecido em WSI fornecem resultados que superam as técnicas clássicas e as do estado da arte.

show abstract

A Systematic Review of Echo State Networks From Design to Application

Cited by 18 publications

References 108 publications

Exploring reservoir computing: Implementation via double stochastic nanowire networks

Exploring reservoir computing: Implementation via double stochastic nanowire networks

Learning Reservoir Dynamics with Temporal Self-Modulation

A Modified Echo State Network Model Using Non-Random Topology

Contact Info

Product

Resources

About