Evolutionary Optimization of Liquid State Machines for Robust Learning

Zhou, Yan; Jin, Yaochu; Ding, Jinliang

doi:10.1007/978-3-030-22796-8_41

Cited by 5 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Particularly, [99] proposed an evolutionary algorithm to optimize the number of neurons and percentage connectivity on a single liquid. Meanwhile, [100] used a covariance matrix adaptation evolution strategy to optimize three parameters, i.e., percentage connectivity, weight distribution and membrane time constant in one liquid. However, [101] pointed out that the above algorithms "only perform parameter optimization in a single liquid and do not optimize the architectures of LSM," and proposed a Neural Architecture Search (NAS) based framework to optimize both architecture and parameters of LSM model.…”

Section: Recent Trends Of Lsm-based Rcmentioning

confidence: 99%

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Zhang

Vargas

2023

IEEE Access

View full text Add to dashboard Cite

Reservoir computing (RC), first applied to temporal signal processing, is a recurrent neural network in which neurons are randomly connected. Once initialized, the connection strengths remain unchanged. Such a simple structure turns RC into a non-linear dynamical system that maps lowdimensional inputs into a high-dimensional space. The model's rich dynamics, linear separability, and memory capacity then enable a simple linear readout to generate adequate responses for various applications. RC spans areas far beyond machine learning, since it has been shown that the complex dynamics can be realized in various physical hardware implementations and biological devices. This yields greater flexibility and shorter computation time. Moreover, the neuronal responses triggered by the model's dynamics shed light on understanding brain mechanisms that also exploit similar dynamical processes. While the literature on RC is vast and fragmented, here we conduct a unified review of RC's recent developments from machine learning to physics, biology, and neuroscience. We first review the early RC models, and then survey the state-of-the-art models and their applications. We further introduce studies on modeling the brain's mechanisms by RC. Finally, we offer new perspectives on RC development, including reservoir design, coding frameworks unification, physical RC implementations, and interaction between RC, cognitive neuroscience and evolution.

show abstract

Section: Recent Trends Of Lsm-based Rcmentioning

confidence: 99%

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Zhang

Vargas

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The parameters of the rest of recurrent neurons (the reservoir are randomly initialized subject to some stability constraints, and kept fixed while the readout layer is trained [227]. Some works have been reported in the last couple of years dealing with the optimization of Reservoir Computing models, such as the composition of the reservoir, connectivity and hierarchical structure of Echo State Networks via Genetic Algorithms [228], or the structural hyper-parameter optimization of Liquid State Machines [229,230] and Echo State Networks [231] using an adapted version of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) solver. The relatively recent advent of Deep versions of Reservoir Computing models [232] unfolds an interesting research playground over which to propose new bio-inspired solvers for topology and hyperparameter optimization.…”

Section: Optimization Of New Deep Learning Architecturesmentioning

confidence: 99%

Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges

Martinez¹,

Ser²,

Villar-Rodríguez³

et al. 2020

Preprint

View full text Add to dashboard Cite

Much has been said about the fusion of bio-inspired optimization algorithms and Deep Learning models for several purposes: from the discovery of network topologies and hyperparametric configurations with improved performance for a given task, to the optimization of the models parameters as a replacement for gradient-based solvers. Indeed, the literature is rich in proposals showcasing the application of assorted nature-inspired approaches for these tasks. In this work we comprehensively review and critically examine contributions made so far based on three axes, each addressing a fundamental question in this research avenue: a) optimization and taxonomy (Why?), including a historical perspective, definitions of optimization problems in Deep Learning, and a taxonomy associated with an in-depth analysis of the literature, b) critical methodological analysis (How?), which together with two case studies, allows us to address learned lessons and recommendations for good practices following the analysis of the literature, and c) challenges and new directions of research (What can be done, and what for?). In summary, three axes -optimization and taxonomy, critical analysis, and challenges -which outline a complete vision of a merger of two technologies drawing up an exciting future for this area of fusion research.

show abstract

“…Most existing evolutionary LSMs predominantly concentrate on parameter optimization, such as liquid density 34 , 35 and liquid size, 36 often resulting in inefficiency. An evolutionary framework of a three-step search is introduced, 35 including architectural parameters such as multiple-liquid architecture, liquid density, excitatory neuron ratio, and so forth.…”

Section: Introductionmentioning

confidence: 99%

“… 37 Some studies respectively apply covariance matrix adaptive evolutionary strategy (CMA-ES) and differential evolution algorithm (DE) to optimize the topology and parameters of the reservoir. 34 , 38 Other NAS-based SNN models aim to maximize classification accuracy with limited computing resources. For instance, energy-efficient SNN architectures 39 are evolved for both classification accuracy and the number of spikes.…”

Section: Introductionmentioning

confidence: 99%

Emergence of brain-inspired small-world spiking neural network through neuroevolution

Pan,

Zhao,

Han

et al. 2024

iScience

View full text Add to dashboard Cite

Evolutionary Optimization of Liquid State Machines for Robust Learning

Cited by 5 publications

References 17 publications

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges

Emergence of brain-inspired small-world spiking neural network through neuroevolution

Contact Info

Product

Resources

About