Evolving the Topology of Large Scale Deep Neural Networks

Assunção, Filipe; Lourenço, Nuno; Machado, Penousal; Ribeiro, Bernardete

doi:10.1007/978-3-319-77553-1_2

Cited by 34 publications

(36 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…CoDeepNEAT [3] is an extension of the NEAT algorithm, which is dedicated to the evolving network structure and hyperparameters of a DNN. • DENSER Assunccao et al [7,8] proposed a two-level representation. The outer level, i.e., GA-level, encodes the general structure of the network and is responsible for representing the sequence of layers.…”

Section: • Codeepneatmentioning

confidence: 99%

Application of evolutionary and swarm optimization in computer vision: a literature survey

Nakane

Bold

Sun

et al. 2020

IPSJ T Comput Vis Appl

View full text Add to dashboard Cite

Evolutionary algorithms (EAs) and swarm algorithms (SAs) have shown their usefulness in solving combinatorial and NP-hard optimization problems in various research fields. However, in the field of computer vision, related surveys have not been updated during the last decade. In this study, inspired by the recent development of deep neural networks in computer vision, which embed large-scale optimization problems, we first describe a literature survey conducted to compensate for the lack of relevant research in this area. Specifically, applications related to the genetic algorithm and differential evolution from EAs, as well as particle swarm optimization and ant colony optimization from SAs and their variants, are mainly considered in this survey.

show abstract

Section: • Codeepneatmentioning

confidence: 99%

Application of evolutionary and swarm optimization in computer vision: a literature survey

Nakane

Bold

Sun

et al. 2020

IPSJ T Comput Vis Appl

View full text Add to dashboard Cite

show abstract

“…In this paper, we are particularly interested in the use of neuroevolution to automate the design of the network architecture and its parameters. This automation is even more relevant for bigger models such as deep neural networks, which produces large search spaces [2,16].…”

Section: Evolutionary Algorithmsmentioning

confidence: 99%

Coevolution of Generative Adversarial Networks

Costa

Lourenço

Machado

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Generative adversarial networks (GAN) present state-of-the-art results in the generation of samples following the distribution of the input dataset. However, GANs are difficult to train, and several aspects of the model should be previously designed by hand. Neuroevolution is a well-known technique used to provide the automatic design of network architectures which was recently expanded to deep neural networks. COEGAN is a model that uses neuroevolution and coevolution in the GAN training algorithm to provide a more stable training method and the automatic design of neural network architectures. COEGAN makes use of the adversarial aspect of the GAN components to implement coevolutionary strategies in the training algorithm. Our proposal was evaluated in the Fashion-MNIST and MNIST dataset. We compare our results with a baseline based on DCGAN and also with results from a random search algorithm. We show that our method is able to discover efficient architectures in the Fashion-MNIST and MNIST datasets. The results also suggest that COEGAN can be used as a training algorithm for GANs to avoid common issues, such as the mode collapse problem.

show abstract

“…Recently, the use of GA in deep learning hyperparameter optimization has been addressed by many researchers. In [4], a GA is used to evolve topologies of large-scale DNN. An encoding schema, based on the configuration of each DNN layer, is proposed.…”

Section: Background and Related Workmentioning

confidence: 99%

“…With the addition of more layers and more processing units in each layer, DNN are able to achieve significant performance in problems of increasing complexity [3]. DNN have presented impressive results, specially in classification and regression applications, including image recognition and computer vision [4]. e design of neural network topologies depends on previous domain knowledge and expertise.…”

Section: Introductionmentioning

confidence: 99%

An Experiment on the Use of Genetic Algorithms for Topology Selection in Deep Learning

Mattioli

Caetano

Cardoso

et al. 2019

Journal of Electrical and Computer Engineering

View full text Add to dashboard Cite

The choice of a good topology for a deep neural network is a complex task, essential for any deep learning project. This task normally demands knowledge from previous experience, as the higher amount of required computational resources makes trial and error approaches prohibitive. Evolutionary computation algorithms have shown success in many domains, by guiding the exploration of complex solution spaces in the direction of the best solutions, with minimal human intervention. In this sense, this work presents the use of genetic algorithms in deep neural networks topology selection. The evaluated algorithms were able to find competitive topologies while spending less computational resources when compared to state-of-the-art methods.

show abstract

Evolving the Topology of Large Scale Deep Neural Networks

Cited by 34 publications

References 23 publications

Application of evolutionary and swarm optimization in computer vision: a literature survey

Application of evolutionary and swarm optimization in computer vision: a literature survey

Coevolution of Generative Adversarial Networks

An Experiment on the Use of Genetic Algorithms for Topology Selection in Deep Learning

Contact Info

Product

Resources

About