A survey of swarm and evolutionary computing approaches for deep learning

Darwish, Ashraf; Hassanien, Aboul Ella; Das, Swagatam

doi:10.1007/s10462-019-09719-2

Cited by 129 publications

(71 citation statements)

References 146 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the traditional approaches are not well suited for optimizing DNNs due to the complex network architectures and large quantities of connection weights. EA-based NAS approaches to optimizing deep network architectures have started gaining momentum again recently [66,67], mainly because they can simultaneously explore multiple areas of the search space and their relative insensitiveness to a local minimum [68,69]. Figure 7 shows a generic framework of EA-based NAS algorithms.…”

Section: Nas Based On Easmentioning

confidence: 99%

From federated learning to federated neural architecture search: a survey

Zhu

Zhang

Jin

2021

Complex Intell. Syst.

102

View full text Add to dashboard Cite

Federated learning is a recently proposed distributed machine learning paradigm for privacy preservation, which has found a wide range of applications where data privacy is of primary concern. Meanwhile, neural architecture search has become very popular in deep learning for automatically tuning the architecture and hyperparameters of deep neural networks. While both federated learning and neural architecture search are faced with many open challenges, searching for optimized neural architectures in the federated learning framework is particularly demanding. This survey paper starts with a brief introduction to federated learning, including both horizontal, vertical, and hybrid federated learning. Then neural architecture search approaches based on reinforcement learning, evolutionary algorithms and gradient-based are presented. This is followed by a description of federated neural architecture search that has recently been proposed, which is categorized into online and offline implementations, and single- and multi-objective search approaches. Finally, remaining open research questions are outlined and promising research topics are suggested.

show abstract

Section: Nas Based On Easmentioning

confidence: 99%

From federated learning to federated neural architecture search: a survey

Zhu

Zhang

Jin

2021

Complex Intell. Syst.

102

View full text Add to dashboard Cite

show abstract

“…This strategy is very common in machine learning for optimizing models during the training process. [32][33][34] bestIndividual ) i 8: end if 9: end for 10: return bestIndividual Hence, the individual codification shown in Figure 1 has been implemented to apply CVOA to optimize deep neural network architectures.…”

Section: Cvoa: Coronavirus Optimization Algorithmmentioning

confidence: 99%

Coronavirus Optimization Algorithm: A Bioinspired Metaheuristic Based on the COVID-19 Propagation Model

et al. 2020

View full text Add to dashboard Cite

This study proposes a novel bioinspired metaheuristic simulating how the coronavirus spreads and infects healthy people. From a primary infected individual (patient zero), the coronavirus rapidly infects new victims, creating large populations of infected people who will either die or spread infection. Relevant terms such as reinfection probability, super-spreading rate, social distancing measures, or traveling rate are introduced into the model to simulate the coronavirus activity as accurately as possible. The infected population initially grows exponentially over time, but taking into consideration social isolation measures, the mortality rate, and number of recoveries, the infected population gradually decreases. The coronavirus optimization algorithm has two major advantages when compared with other similar strategies. First, the input parameters are already set according to the disease statistics, preventing researchers from initializing them with arbitrary values. Second, the approach has the ability to end after several iterations, without setting this value either. Furthermore, a parallel multivirus version is proposed, where several coronavirus strains evolve over time and explore wider search space areas in less iterations. Finally, the metaheuristic has been combined with deep learning models, to find optimal hyperparameters during the training phase. As application case, the problem of electricity load time series forecasting has been addressed, showing quite remarkable performance.

show abstract

“…The evolutionary computing (EC) and swarm intelligence (SI) are mature global optimization methods with high robustness and wide applicability, which have the characteristics of self-organization, self-adaptation, and self-learning [ 31 ]. Moreover, the EC and SI algorithms are not limited by the nature of the problem and can effectively deal with complex optimization problems that are difficult to solve by traditional optimization algorithm.…”

Section: Introductionmentioning

confidence: 99%

Sonar Objective Detection Based on Dilated Separable Densely Connected CNNs and Quantum-Behaved PSO Algorithm

Zhen

Wang

Guo

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Underwater sonar objective detection plays an important role in the field of ocean exploration. In order to solve the problem of sonar objective detection under the complex environment, a sonar objective detection method is proposed based on dilated separable densely connected convolutional neural networks (DS-CNNs) and quantum-behaved particle swarm optimization (QPSO) algorithm. Firstly, the dilated separable convolution kernel is proposed to extend the local receptive field and enhance the feature extraction ability of the convolution layers. Secondly, based on the linear interpolation algorithm, a multisampling pooling (MS-pooling) operation is proposed to reduce the feature information loss and restore image resolution. At last, with contraction-expansion factor and difference variance in the traditional particle swarm optimization algorithm introduced, the QPSO algorithm is employed to optimize the weight parameters of the network model. The proposed method is validated on the sonar image dataset and is compared with other existing methods. Using DS-CNNs to detect different kinds of sonar objectives, the experiments shows that the detection accuracy of DS-CNNs reaches 96.98% and DS-CNNs have better detection effect and stronger robustness.

show abstract

A survey of swarm and evolutionary computing approaches for deep learning

Cited by 129 publications

References 146 publications

From federated learning to federated neural architecture search: a survey

From federated learning to federated neural architecture search: a survey

Coronavirus Optimization Algorithm: A Bioinspired Metaheuristic Based on the COVID-19 Propagation Model

Sonar Objective Detection Based on Dilated Separable Densely Connected CNNs and Quantum-Behaved PSO Algorithm

Contact Info

Product

Resources

About