A Survey on Evolutionary Neural Architecture Search

Liu, Yuqiao; Sun, Yanan; Xue, Bing; Zhang, Mengjie; Yen, Gary G.; Tan, Kay Chen

doi:10.48550/arxiv.2008.10937

Cited by 10 publications

(10 citation statements)

References 153 publications

(376 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar idea is also adopted for accelerating ENAS in [46] and [47]. Despite that the idea of early stopping can considerably reduce the computational cost of ENAS approaches, it easily leads to inaccurate estimation on individual quality, especially for complicated architectures as stated in [48]. This makes it hard for early stopping based ENAS approaches to achieve competitive performance in NAS.…”

Section: B Related Workmentioning

confidence: 99%

“…In [17], Sun et al utilized a hashing method to record both architecture information and fitness of each individual, so that the fitness value for an individual can be directly obtained if this individual has been recorded. Despite avoiding redundant individual evaluations in ENAS, these strategies still consume expensive computational cost to achieve high-quality architectures [48].…”

Section: B Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation

Yang¹,

Tian²,

Xiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Evolutionary neural architecture search (ENAS) has recently received increasing attention by effectively finding high-quality neural architectures, which however consumes high computational cost by training the architecture encoded by each individual for complete epochs in individual evaluation. Numerous ENAS approaches have been developed to reduce the evaluation cost, but it is often difficult for most of these approaches to achieve high evaluation accuracy. To address this issue, in this paper we propose an accelerated ENAS via multifidelity evaluation termed MFENAS, where the individual evaluation cost is significantly reduced by training the architecture encoded by each individual for only a small number of epochs. The balance between evaluation cost and evaluation accuracy is well maintained by suggesting a multi-fidelity evaluation, which identifies the potentially good individuals that cannot survive from previous generations by integrating multiple evaluations under different numbers of training epochs. For high diversity of neural architectures, a population initialization strategy is devised to produce different neural architectures varying from ResNetlike architectures to Inception-like ones. Experimental results on CIFAR-10 show that the architecture obtained by the proposed MFENAS achieves a 2.39% test error rate at the cost of only 0.6 GPU days on one NVIDIA 2080TI GPU, demonstrating the superiority of the proposed MFENAS over state-of-the-art NAS approaches in terms of both computational cost and architecture quality. The architecture obtained by the proposed MFENAS is then transferred to CIFAR-100 and ImageNet, which also exhibits competitive performance to the architectures obtained by existing NAS approaches. The source code of the proposed MFENAS is available at https://github.com/DevilYangS/MFENAS/.

show abstract

Section: B Related Workmentioning

confidence: 99%

Section: B Related Workmentioning

confidence: 99%

Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation

Yang¹,

Tian²,

Xiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To have a real impact, however, the resources involved are colossal -as seen in many papers, they are on the scale of hundreds to thousands of GPUs. Recent studies [9,17,18] even point out that EA and RL may not outperform random search. These drawbacks have led to the development of another paradigm: the idea that a better problem formulation could partially alleviate the hardware constraint and provide a greater chance of success in finding a good architecture.…”

Section: Mutating (Random Information Perturbation)mentioning

confidence: 99%

Neural Architecture Search in operational context: a remote sensing case-study

Cazasnoves,

Ganaye,

Sanchis

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep learning has become in recent years a cornerstone tool fueling key innovations in the industry, such as autonomous driving. To attain good performances, the neural network architecture used for a given application must be chosen with care. These architectures are often handcrafted and therefore prone to human biases and sub-optimal selection. Neural Architecture Search (NAS) is a framework introduced to mitigate such risks by jointly optimizing the network architectures and its weights. Albeit its novelty, it was applied on complex tasks with significant results -e.g. semantic image segmentation. In this technical paper, we aim to evaluate its ability to tackle a challenging operational task: semantic segmentation of objects of interest in satellite imagery. Designing a NAS framework is not trivial and has strong dependencies to hardware constraints. We therefore motivate our NAS approach selection and provide corresponding implementation details. We also present novel ideas to carry out other such use-case studies.

show abstract

“…Several recent design spaces for NAS have composed generic neural network functional components while preserving inductive priors from successful state-of-the-art computer vision or natural language processing neural network architectures (Liu et al, 2020). On the other hand, as it applies to dynamical system identification, NAS with neuroevolutionary methods have typically used lower level components such as neurons and basic function compositions applied in black-box systems modeling without the benefit of physics-modeling based priors embedded in the design space (Al-Mahasneh et al, 2017;Subudhi and Jena, 2011a,b;Yang et al, 2017;Ferariu and Burlacu, 2011;Ayala et al, 2020;Hatanaka et al, 2006;Yang et al, 2017;Gaier and Ha, 2019).…”

Section: Introductionmentioning

confidence: 99%

Physics-Informed Neural State Space Models via Learning and Evolution

Skomski¹,

Drgoňa²,

Tuor³

2020

Preprint

View full text Add to dashboard Cite

Recent works exploring deep learning application to dynamical systems modeling have demonstrated that embedding physical priors into neural networks can yield more effective, physicallyrealistic, and data-efficient models. However, in the absence of complete prior knowledge of a dynamical system's physical characteristics, determining the optimal structure and optimization strategy for these models can be difficult. In this work, we explore methods for discovering neural state space dynamics models for system identification. Starting with a design space of block-oriented state space models and structured linear maps with strong physical priors, we encode these components into a model genome alongside network structure, penalty constraints, and optimization hyperparameters. Demonstrating the overall utility of the design space, we employ an asynchronous genetic search algorithm that alternates between model selection and optimization and obtains accurate physically consistent models of three physical systems: an aerodynamics body, a continuous stirred tank reactor, and a two tank interacting system.

show abstract

A Survey on Evolutionary Neural Architecture Search

Cited by 10 publications

References 153 publications

Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation

Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation

Neural Architecture Search in operational context: a remote sensing case-study

Physics-Informed Neural State Space Models via Learning and Evolution

Contact Info

Product

Resources

About