Surrogate-Assisted Evolutionary Search of Spiking Neural Architectures in Liquid State Machines

Zhou, Yan; Jin, Yaochu; Ding, Jinliang

doi:10.1016/j.neucom.2020.04.079

Cited by 30 publications

(17 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The parameters of the rest of recurrent neurons (the reservoir are randomly initialized subject to some stability constraints, and kept fixed while the readout layer is trained [227]. Some works have been reported in the last couple of years dealing with the optimization of Reservoir Computing models, such as the composition of the reservoir, connectivity and hierarchical structure of Echo State Networks via Genetic Algorithms [228], or the structural hyper-parameter optimization of Liquid State Machines [229,230] and Echo State Networks [231] using an adapted version of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) solver. The relatively recent advent of Deep versions of Reservoir Computing models [232] unfolds an interesting research playground over which to propose new bio-inspired solvers for topology and hyperparameter optimization.…”

Section: Optimization Of New Deep Learning Architecturesmentioning

confidence: 99%

Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges

Martinez¹,

Ser²,

Villar-Rodríguez³

et al. 2020

Preprint

View full text Add to dashboard Cite

Much has been said about the fusion of bio-inspired optimization algorithms and Deep Learning models for several purposes: from the discovery of network topologies and hyperparametric configurations with improved performance for a given task, to the optimization of the models parameters as a replacement for gradient-based solvers. Indeed, the literature is rich in proposals showcasing the application of assorted nature-inspired approaches for these tasks. In this work we comprehensively review and critically examine contributions made so far based on three axes, each addressing a fundamental question in this research avenue: a) optimization and taxonomy (Why?), including a historical perspective, definitions of optimization problems in Deep Learning, and a taxonomy associated with an in-depth analysis of the literature, b) critical methodological analysis (How?), which together with two case studies, allows us to address learned lessons and recommendations for good practices following the analysis of the literature, and c) challenges and new directions of research (What can be done, and what for?). In summary, three axes -optimization and taxonomy, critical analysis, and challenges -which outline a complete vision of a merger of two technologies drawing up an exciting future for this area of fusion research.

show abstract

Section: Optimization Of New Deep Learning Architecturesmentioning

confidence: 99%

Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges

Martinez¹,

Ser²,

Villar-Rodríguez³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…To cope with the state-of-the-art results in real world applications, many efforts on enhancing the LSM performance were based on the exploration of different LSM topologies [17], [18], new training algorithms or cost-intensive parameters search [19], [20], which increase accuracy at the cost of performance or resource overhead. The input format is another issue that has significant impact on the performance of the LSM.…”

Section: Introductionmentioning

confidence: 99%

A human vision based system for biometric images recognition

Boukhari

Benyettou

Belmadani

2022

IJEECS

View full text Add to dashboard Cite

<p><span>In this paper, a universal biometric system based on human vision is proposed. From recent biological and physiological results, A human identification system that approximates the natural vision and recognition of individuals is conceived. Liquid state machine (LSM), as a recurrent spiking neural network, is highly inspired by the brain neural architecture with low training cost. However, input dimension of large scale images requires efficient processing at the cost of performance or resource overhead. This paper propose a new neural input coding for images based on frequency signals rather than pixels. Each image is filtered and fragmented then the LSM liquid (or reservoir) will receive, first, high frequency signals, then low frequency signals from each fragment. The two sets of output neurons states corresponding to each type of filter will be matched to the entire enrollment database. A weighted sum rule between the matching results will determine the right class of a biometric image. The system was tested on three different biometric datasets: face, palmprint and off-line signature, results show the reliability of the proposed approach.</span></p>

show abstract

“…It is well recognized, however, that most MOEAs require a large number of function evaluations before a set of diverse and well-converged non-dominated solutions can be found, which makes it hard for them to be directly applied to solve a class of data-driven optimization problems [24], whose objectives can be evaluated by means of conducting time-consuming computer simulations or costly physical experiments. Examples of real-world data-driven multi-objective optimization problems include blast furnace optimization [7], air intake ventilation system optimization [10], airfoil design optimization [29], bridge design optimization [34], design optimization for a stiffened cylindrical shell with variable ribs [57], optimization of operation of crude oil distillation units [19], resource allocation of trauma systems [47], and also neural architecture search [42,58], among many others.…”

Section: Introductionmentioning

confidence: 99%

A federated data-driven evolutionary algorithm for expensive multi-/many-objective optimization

Jin

2021

Complex Intell. Syst.

Self Cite

View full text Add to dashboard Cite

Data-driven optimization has found many successful applications in the real world and received increased attention in the field of evolutionary optimization. Most existing algorithms assume that the data used for optimization are always available on a central server for construction of surrogates. This assumption, however, may fail to hold when the data must be collected in a distributed way and are subject to privacy restrictions. This paper aims to propose a federated data-driven evolutionary multi-/many-objective optimization algorithm. To this end, we leverage federated learning for surrogate construction so that multiple clients collaboratively train a radial-basis-function-network as the global surrogate. Then a new federated acquisition function is proposed for the central server to approximate the objective values using the global surrogate and estimate the uncertainty level of the approximated objective values based on the local models. The performance of the proposed algorithm is verified on a series of multi-/many-objective benchmark problems by comparing it with two state-of-the-art surrogate-assisted multi-objective evolutionary algorithms.

show abstract

Surrogate-Assisted Evolutionary Search of Spiking Neural Architectures in Liquid State Machines

Cited by 30 publications

References 32 publications

Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges

Lights and Shadows in Evolutionary Deep Learning: Taxonomy, Critical Methodological Analysis, Cases of Study, Learned Lessons, Recommendations and Challenges

A human vision based system for biometric images recognition

A federated data-driven evolutionary algorithm for expensive multi-/many-objective optimization

Contact Info

Product

Resources

About