Emergence of brain-inspired small-world spiking neural network through neuroevolution

Pan, Wenxuan; Zhao, Feifei; Han, Bing; Dong, Yiting; Zeng, Yi

doi:10.1016/j.isci.2024.108845

Cited by 1 publication

(1 citation statement)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Randomly generated or simple recurrent connections may not effectively optimize RSNNs' performance. Recently, Pan et al ( 2024 ) introduced a multi-objective Evolutionary Liquid State Machine (ELSM) inspired by neuroevolution process. Chakraborty and Mukhopadhyay ( 2023 ) proposed Heterogeneous recurrent spiking neural network (HRSNN), in which recurrent layers are composed of heterogeneous neurons with different dynamics.…”

Section: Introductionmentioning

confidence: 99%

Composing recurrent spiking neural networks using locally-recurrent motifs and risk-mitigating architectural optimization

Zhang,

Geng,

2024

Front. Neurosci.

View full text Add to dashboard Cite

In neural circuits, recurrent connectivity plays a crucial role in network function and stability. However, existing recurrent spiking neural networks (RSNNs) are often constructed by random connections without optimization. While RSNNs can produce rich dynamics that are critical for memory formation and learning, systemic architectural optimization of RSNNs is still an open challenge. We aim to enable systematic design of large RSNNs via a new scalable RSNN architecture and automated architectural optimization. We compose RSNNs based on a layer architecture called Sparsely-Connected Recurrent Motif Layer (SC-ML) that consists of multiple small recurrent motifs wired together by sparse lateral connections. The small size of the motifs and sparse inter-motif connectivity leads to an RSNN architecture scalable to large network sizes. We further propose a method called Hybrid Risk-Mitigating Architectural Search (HRMAS) to systematically optimize the topology of the proposed recurrent motifs and SC-ML layer architecture. HRMAS is an alternating two-step optimization process by which we mitigate the risk of network instability and performance degradation caused by architectural change by introducing a novel biologically-inspired “self-repairing” mechanism through intrinsic plasticity. The intrinsic plasticity is introduced to the second step of each HRMAS iteration and acts as unsupervised fast self-adaptation to structural and synaptic weight modifications introduced by the first step during the RSNN architectural “evolution.” We demonstrate that the proposed automatic architecture optimization leads to significant performance gains over existing manually designed RSNNs: we achieve 96.44% on TI46-Alpha, 94.66% on N-TIDIGITS, 90.28% on DVS-Gesture, and 98.72% on N-MNIST. To the best of the authors' knowledge, this is the first work to perform systematic architecture optimization on RSNNs.

show abstract

Section: Introductionmentioning

confidence: 99%