EEG feature learning with Intrinsic Plasticity based Deep Echo State Network

Fourati, Rahma; Ammar, Boudour; Jin, Yaochu; Alimi, Adel M.

doi:10.1109/ijcnn48605.2020.9207464

Cited by 8 publications

(4 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Zhang et al ( 2019 ), the application of IP mechanism significantly improves computational performance in terms of learning speed, accuracy, and robustness to input variations and noise. Fourati et al ( 2020 ) proposes a deep echo state network that utilizes intrinsic plasticity to drive reservoir neuron activities to follow a desired Gaussian distribution, enabling the learning of discriminative EEG representations and demonstrating its effectiveness on emotion recognition benchmarks. Zhang et al ( 2020 ) proposes a novel IP learning rule based on a soft-reset spiking neuron model, which ensures the neuron's membrane potential is mathematically continuous and differentiable.…”

Section: Methodsmentioning

confidence: 99%

Composing recurrent spiking neural networks using locally-recurrent motifs and risk-mitigating architectural optimization

Zhang,

Geng,

2024

Front. Neurosci.

View full text Add to dashboard Cite

In neural circuits, recurrent connectivity plays a crucial role in network function and stability. However, existing recurrent spiking neural networks (RSNNs) are often constructed by random connections without optimization. While RSNNs can produce rich dynamics that are critical for memory formation and learning, systemic architectural optimization of RSNNs is still an open challenge. We aim to enable systematic design of large RSNNs via a new scalable RSNN architecture and automated architectural optimization. We compose RSNNs based on a layer architecture called Sparsely-Connected Recurrent Motif Layer (SC-ML) that consists of multiple small recurrent motifs wired together by sparse lateral connections. The small size of the motifs and sparse inter-motif connectivity leads to an RSNN architecture scalable to large network sizes. We further propose a method called Hybrid Risk-Mitigating Architectural Search (HRMAS) to systematically optimize the topology of the proposed recurrent motifs and SC-ML layer architecture. HRMAS is an alternating two-step optimization process by which we mitigate the risk of network instability and performance degradation caused by architectural change by introducing a novel biologically-inspired “self-repairing” mechanism through intrinsic plasticity. The intrinsic plasticity is introduced to the second step of each HRMAS iteration and acts as unsupervised fast self-adaptation to structural and synaptic weight modifications introduced by the first step during the RSNN architectural “evolution.” We demonstrate that the proposed automatic architecture optimization leads to significant performance gains over existing manually designed RSNNs: we achieve 96.44% on TI46-Alpha, 94.66% on N-TIDIGITS, 90.28% on DVS-Gesture, and 98.72% on N-MNIST. To the best of the authors' knowledge, this is the first work to perform systematic architecture optimization on RSNNs.

show abstract

Section: Methodsmentioning

confidence: 99%

Composing recurrent spiking neural networks using locally-recurrent motifs and risk-mitigating architectural optimization

Zhang,

Geng,

2024

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Medical field is also an important application of ESNs. At present, the main practices include feature learning of vital signs, such as electroencephalogram (EEG)-based feature extraction [125,236,237] and event detection [238,239,240,241,242], electrocardiogram (ECG)-based feature clustering [243], atrial fibrillation detection [244,245], arterial blood pressure prediction [246] and magnetic resonance imaging (MRI)-based disease diagnosis [247]. Meanwhile, there are many methods for disease diagnosis.…”

Section: Real-world Tasks Orientatedmentioning

confidence: 99%

A Review of Designs and Applications of Echo State Networks

Sun¹,

Song²,

Hong³

et al. 2020

Preprint

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) have demonstrated their outstanding ability in sequence tasks and have achieved state-of-the-art in wide range of applications, such as industrial, medical, economic and linguistic. Echo State Network (ESN) is simple type of RNNs and has emerged in the last decade as an alternative to gradient descent training based RNNs. ESN, with a strong theoretical ground, is practical, conceptually simple, easy to implement. It avoids non-converging and computationally expensive in the gradient descent methods. Since ESN was put forward in 2002, abundant existing works have promoted the progress of ESN, and the recently introduced Deep ESN model opened the way to uniting the merits of deep learning and ESNs. Besides, the combinations of ESNs with other machine learning models have also overperformed baselines in some applications. However, the apparent simplicity of ESNs can sometimes be deceptive and successfully applying ESNs needs some experience. Thus, in this paper, we categorize the ESN-based methods to basic ESNs, DeepESNs and combinations, then analyze them from the perspective of theoretical studies, network designs and specific applications. Finally, we discuss the challenges and opportunities of ESNs by summarizing the open questions and proposing possible future works.

show abstract

“…Fourati et al. [4] modelled temporal representation from EEG data with a deep echo state network (DeepESN). Cui et al.…”

Section: Introductionmentioning

confidence: 99%

“…For emotion classification, constructed models are mainly divided into two types: machine learning models and deep learning models. Fourati et al [4] modelled temporal representation from EEG data with a deep echo state network (DeepESN). Cui et al [5] proposed an end-to-end Regional-Asymmetric Convolutional Neural Network (RACNN) to recognize different kinds of emotional states.…”

mentioning

confidence: 99%

A feature‐based on potential and differential entropy information for electroencephalogram emotion recognition

Xie

Chai

et al. 2021

Electronics Letters

View full text Add to dashboard Cite

Electroencephalogram (EEG) signals by virtue of its good time resolution can provide a more comprehensive way for emotion recognition. To further mine the emotion-relevant information of EEG signals, a kind of innovative and effective feature named mean differential entropy (MDE) is proposed for EEG emotion recognition (EER). The MDE feature extracted from EEG signals supplements the additional potential information of each EEG frame on the basis of time-frequency information. To verify its effectiveness, extensive experiments are conducted on DEAP (database for emotion analysis using physiological signals) dataset. The experiment results of the proposed methods on public EEG emotion dataset DEAP reached accuracies of 94.56% and 91.84% in valence dimension and four emotion labels with gradient boosting decision tree (GBDT), and 95.15% in arousal dimension with Random Forest (RF), which proved that the MDE feature is helpful to improve the performance of EER. 62076094, Shanghai Science and Technology Program "Distributed and generative few-shot algorithm and theory research" under Grant No. 20511100600.

show abstract

EEG feature learning with Intrinsic Plasticity based Deep Echo State Network

Cited by 8 publications

References 35 publications

Composing recurrent spiking neural networks using locally-recurrent motifs and risk-mitigating architectural optimization

Composing recurrent spiking neural networks using locally-recurrent motifs and risk-mitigating architectural optimization

A Review of Designs and Applications of Echo State Networks

A feature‐based on potential and differential entropy information for electroencephalogram emotion recognition

Contact Info

Product

Resources

About