A Review of Designs and Applications of Echo State Networks

Sun, C. P.; Song, Moxian; Hong, Shenda; Li, Hongyan

doi:10.48550/arxiv.2012.02974

Cited by 11 publications

(10 citation statements)

References 275 publications

(257 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, we only examine the digital domain implementation. Moreover, we focus on the leaky-ESN, as it is believed to often outperform standard ESNs and is more flexible due to time-scale phenomena [62], [63]. The equations of the leaky-ESN for a certain time step t are given as:…”

Section: F Echo State Networkmentioning

confidence: 99%

Computational Complexity Evaluation of Neural Network Applications in Signal Processing

Freire¹,

Srivallapanondh²,

Napoli³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we provide a systematic approach for assessing and comparing the computational complexity of neural network layers in digital signal processing. We provide and link four software-to-hardware complexity measures, defining how the different complexity metrics relate to the layers' hyperparameters. This paper explains how to compute these four metrics for feed-forward and recurrent layers, and defines in which case we ought to use a particular metric depending on whether we characterize a more soft-or hardware-oriented application. One of the four metrics, called 'the number of additions and bit shifts (NABS)', is newly introduced for heterogeneous quantization. NABS characterizes the impact of not only the bitwidth used in the operation but also the type of quantization used in the arithmetical operations. We intend this work to serve as a baseline for the different levels (purposes) of complexity estimation related to the neural networks' application in real-time digital signal processing, aiming at unifying the computational complexity estimation.

show abstract

Section: F Echo State Networkmentioning

confidence: 99%

Computational Complexity Evaluation of Neural Network Applications in Signal Processing

Freire¹,

Srivallapanondh²,

Napoli³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this paper, we use the concept of leaky-ESN [41] containing no output feedback connections. Our motivation to choose the leaky-ESN architecture is because the initial experimental analysis proved that the leaky-ESN configuration outperforms the traditional ESN in feature extraction for noisy time series [38], and the latter is an important property in optical transmission-related tasks. The leaky-ESN is formalized for a certain time-step t, as follows:…”

Section: Echo State Networkmentioning

confidence: 99%

Performance versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems

Freire,

Osadchuk,

Spinnler

et al. 2021

Preprint

View full text Add to dashboard Cite

We present the results of the comparative analysis of the performance versus complexity for several types of artificial neural networks (NNs) used for nonlinear channel equalization in coherent optical communication systems. The comparison has been carried out using an experimental set-up with transmission dominated by the Kerr nonlinearity and component imperfections. For the first time, we investigate the application to the channel equalization of the convolution layer (CNN) in combination with a bidirectional long short-term memory (biLSTM) layer and the design combining CNN with a multi-layer perceptron. Their performance is compared with the one delivered by the previously proposed NN equalizer models: one biLSTM layer, three-denselayer perceptron, and the echo state network. Importantly, all architectures have been initially optimized by a Bayesian optimizer. We present the derivation of the computational complexity associated with each NN type -in terms of real multiplications per symbol so that these results can be applied to a large number of communication systems. We demonstrated that in the specific considered experimental system the convolutional layer coupled with the biLSTM (CNN+biLSTM) provides the highest Q-factor improvement compared to the reference linear chromatic dispersion compensation (2.9 dB improvement). We examine the tradeoff between the computational complexity and performance of all equalizers and demonstrate that the CNN+biLSTM is the best option when the computational complexity is not constrained, while when we restrict the complexity to lower levels, the threelayer perceptron provides the best performance. Our complexity analysis for different NNs is generic and can be applied in a wide range of physical and engineering systems.

show abstract

“…In this study, we use the numerical implementation of RC called the echo states network (ESN) [ 31 ]. Many applications use the ESN algorithm [ 32 ], but only a few are applied to biomechanics, such as gesture recognition [ 33 ], muscle drive-in actuation [ 34 ] and exoskeleton control [ 35 ]. The ESN model is much simpler to implement on hardware devices compared to most of the advanced GED models recently developed.…”

Section: Introductionmentioning

confidence: 99%

Detecting Gait Events from Accelerations Using Reservoir Computing

Chiasson-Poirier

Younesian²,

Turcot³

et al. 2022

Sensors

View full text Add to dashboard Cite

Segmenting the gait cycle into multiple phases using gait event detection (GED) is a well-researched subject with many accurate algorithms. However, the algorithms that are able to perform accurate and robust GED for real-life environments and physical diseases tend to be too complex for their implementation on simple hardware systems limited in computing power and memory, such as those used in wearable devices. This study focuses on a numerical implementation of a reservoir computing (RC) algorithm called the echo state network (ESN) that is based on simple computational steps that are easy to implement on portable hardware systems for real-time detection. RC is a neural network method that is widely used for signal processing applications and uses a fast-training method based on a ridge regression adapted to the large quantity and variety of IMU data needed to use RC in various real-life environment GED. In this study, an ESN was used to perform offline GED with gait data from IMU and ground force sensors retrieved from three databases for a total of 28 healthy adults and 15 walking conditions. Our main finding is that despite its low complexity, ESN is robust for GED, with performance comparable to other state-of-the-art algorithms. Our results show the ESN is robust enough to obtain good detection results in all conditions if the algorithm is trained with variable data that match those conditions. The distribution of the mean absolute errors (MAE) between the detection times from the ESN and the force sensors were between 40 and 120 ms for 6 defined gait events (95th percentile). We compared our ESN with four different state-of-the-art algorithms from the literature. The ESN obtained a MAE not more than 10 ms above three other reference algorithms for normal walking indoor and outdoor conditions and yielded the 2nd lowest MAE and the 2nd highest true positive rate and specificity when applied to outdoor walking and running conditions. Our work opens the door to using the ESN as a GED for applications in wearable sensors for long-term patient monitoring.

show abstract

A Review of Designs and Applications of Echo State Networks

Cited by 11 publications

References 275 publications

Computational Complexity Evaluation of Neural Network Applications in Signal Processing

Computational Complexity Evaluation of Neural Network Applications in Signal Processing

Performance versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems

Detecting Gait Events from Accelerations Using Reservoir Computing

Contact Info

Product

Resources

About