DENSER: deep evolutionary network structured representation

Assunção, Filipe; Lourenço, Nuno; Machado, Penousal; Ribeiro, Bernardete

doi:10.1007/s10710-018-9339-y

Cited by 99 publications

(56 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…From these results we hypothesise that superior generalisation performances are obtained by incremental development, when passing from more simple to more challenging datasets. That is the reason why there is no statistical difference in the CIFAR-10 vs. MNIST )SVHN )CIFAR-10 setups: the CIFAR-10 is per-se more challenging to solve than the remaining ones, and therefore, as already noticed in a previous article [13], the DANNs generated for addressing CIFAR-10 tend to be able to solve other easier problems. The remarkable aspect of incremental development is when a DANN optimised for Fashion is able to get better results on the CIFAR-10, compared to when the DANNs for Fashion are not evolved in an incremental fashion.…”

Section: Experimental Results: Generalisation Of the Modelsmentioning

confidence: 70%

“…NeuroEvolution (NE) approaches are usually grouped according to the target of evolution, i.e., topology [3,4], learning (i.e., weights, parameters, or learning policies) [5,6,7], or the simultaneous evolution of the topology and learning [8,9]. Nonetheless, more recent efforts have been put towards the proposal of methods that deal with the optimisation DANNs, and thus we feel that it is more intuitive to divide them into small-scale [5,8] and large-scale [7,10,11,12,13] NE. The current paper focuses on the latter; a complete survey can be found in [14].…”

Section: Related Workmentioning

confidence: 99%

“…Fast-DENSER [2] is an extension of Deep Evolutionary Network Structured Representation (DENSER) [13]: a general-purpose grammar-based NE approach for optimising DANNs. DENSER can search for any type of DANN, and the target of evolution is specified in a Context-Free Grammar (CFG).…”

Section: Fast-densermentioning

confidence: 99%

See 2 more Smart Citations

Incremental Evolution and Development of Deep Artificial Neural Networks

Assunção

Lourenço

Ribeiro

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

NeuroEvolution (NE) methods are known for applying Evolutionary Computation to the optimisation of Artificial Neural Networks (ANNs). Despite aiding non-expert users to design and train ANNs, the vast majority of NE approaches disregard the knowledge that is gathered when solving other tasks, i.e., evolution starts from scratch for each problem, ultimately delaying the evolutionary process. To overcome this drawback, we extend Fast Deep Evolutionary Network Structured Representation (Fast-DENSER) to incremental development. We hypothesise that by transferring the knowledge gained from previous tasks we can attain superior results and speedup evolution. The results show that the average performance of the models generated by incremental development is statistically superior to the non-incremental average performance. In case the number of evaluations performed by incremental development is smaller than the performed by non-incremental development the attained results are similar in performance, which indicates that incremental development speeds up evolution. Lastly, the models generated using incremental development generalise better, and thus, without further evolution, report a superior performance on unseen problems.

show abstract

Section: Experimental Results: Generalisation Of the Modelsmentioning

confidence: 70%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Incremental Evolution and Development of Deep Artificial Neural Networks

Assunção

Lourenço

Ribeiro

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…In order to design the Artificial Neural Networks (ANN) automatically with evolutionary Fig. 10: A Generative Adversarial Network Architecture computation a Deep Evolutionary Network Structured Representation (DENSER) was proposed in [121], where the optimal design for the network is achieved by a bi-leveled representation. The outer level deals with the number of layers and their sequence whereas the inner layer optimizes the parameters and hyper parameters associated with each layer defined by a context-free human perceivable grammar.…”

Section: Swarm Intelligence In Deep Learningmentioning

confidence: 99%

A review of deep learning with special emphasis on architectures, applications and recent trends

Sengupta

Basak

Saikia

et al. 2020

Knowledge-Based Systems

303

143

View full text Add to dashboard Cite

Deep learning has solved a problem that as little as five years ago was thought by many to be intractable -the automatic recognition of patterns in data; and it can do so with an accuracy that often surpasses that of human beings. It has solved problems beyond the realm of traditional, hand-crafted machine learning algorithms and captured the imagination of practitioners trying to make sense out of the flood of data that now inundates our society. As public awareness of the efficacy of deep learning increases so does the desire to make use of it. But even for highly trained professionals it can be daunting to approach the rapidly increasing body of knowledge produced by experts in the field. Where does one start? How does one determine if a particular Deep Learning model is applicable to their problem? How does one train and deploy such a network? A primer on the subject can be a good place to start. With that in mind, we present an overview of some of the key multilayer artificial neural networks that comprise deep learning. We also discuss some new automatic architecture optimization protocols that use multi-agent approaches. Further, since guaranteeing system uptime is becoming critical to many computer applications, we include a section on using neural networks for fault detection and subsequent mitigation. This is followed by an exploratory survey of several application areas where deep learning has emerged as a game-changing technology: anomalous behavior detection in financial applications or in financial time-series forecasting, predictive and prescriptive analytics, medical image processing and analysis and power systems research. The thrust of this review is to outline emerging areas of application-oriented research within the deep learning community as well as to provide a handy reference to researchers seeking to use deep learning in their work for what it does best: statistical pattern recognition with unparalleled learning capacity with the ability to scale with information.

show abstract

“…Deep Evolutionary Network Structured Representation (DENSER) [20], is a general-purpose grammar-based Neu-roEvolution (NE) approach. It has successfully been applied in object detection tasks, and all the user inputs are defined in a human-readable format, and thus the framework is easy to adapt to different domains and network structures.…”

Section: Deep Evolutionary Network Structured Representationmentioning

confidence: 99%

Automatic Design of Artificial Neural Networks for Gamma-Ray Detection

Assunção

Correia

Conceição³

et al. 2019

IEEE Access

Self Cite

View full text Add to dashboard Cite

The goal of this work is to investigate the possibility of improving current gamma/hadron discrimination based on their shower patterns recorded on the ground. To this end we propose the use of Convolutional Neural Networks (CNNs) for their ability to distinguish patterns based on automatically designed features. In order to promote the creation of CNNs that properly uncover the hidden patterns in the data, and at same time avoid the burden of hand-crafting the topology and learning hyper-parameters we resort to NeuroEvolution; in particular we use Fast-DENSER++, a variant of Deep Evolutionary Network Structured Representation. The results show that the best CNN generated by Fast-DENSER++ improves by a factor of 2 when compared with the results reported by classic statistical approaches. Additionally, we experiment ensembling the 10 best generated CNNs, one from each of the evolutionary runs; the ensemble leads to an improvement by a factor of 2.3. These results show that it is possible to improve the gamma/hadron discrimination based on CNNs that are automatically generated and are trained with instances of the ground impact patterns.

show abstract

DENSER: deep evolutionary network structured representation

Cited by 99 publications

References 56 publications

Incremental Evolution and Development of Deep Artificial Neural Networks

Incremental Evolution and Development of Deep Artificial Neural Networks

A review of deep learning with special emphasis on architectures, applications and recent trends

Automatic Design of Artificial Neural Networks for Gamma-Ray Detection

Contact Info

Product

Resources

About