Auto-creation of Effective Neural Network Architecture by Evolutionary Algorithm and ResNet for Image Classification

Chen, Zefeng; Zhou, Yuren; Huang, Zhengxin

doi:10.1109/smc.2019.8914267

Cited by 20 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evolutionary methodologies [9,30,38,39,52] utilize genetic algorithms to discover the efficient neural architecture in a large search space. Evolutionary algorithms have also been successfully deployed for CNN optimizations, such as ,for compression [25], pruning [19] and hyperparameter optimization [45].…”

Section: Related Workmentioning

confidence: 99%

Designing convolutional neural networks with constrained evolutionary piecemeal training

Sapra

Pimentel

2021

Appl Intell

View full text Add to dashboard Cite

The automated architecture search methodology for neural networks is known as Neural Architecture Search (NAS). In recent times, Convolutional Neural Networks (CNNs) designed through NAS methodologies have achieved very high performance in several fields, for instance image classification and natural language processing. Our work is in the same domain of NAS, where we traverse the search space of neural network architectures with the help of an evolutionary algorithm which has been augmented with a novel approach of piecemeal-training. In contrast to the previously published NAS techniques, wherein the training with given data is considered an isolated task to estimate the performance of neural networks, our work demonstrates that a neural network architecture and the related weights can be jointly learned by combining concepts of the traditional training process and evolutionary architecture search in a single algorithm. The consolidation has been realised by breaking down the conventional training technique into smaller slices and collating them together with an integrated evolutionary architecture search algorithm. The constraints on architecture search space are placed by limiting its various parameters within a specified range of values, consequently regulating the neural network’s size and memory requirements. We validate this concept on two vastly different datasets, namely, the CIFAR-10 dataset in the domain of image classification, and PAMAP2 dataset in the Human Activity Recognition (HAR) domain. Starting from randomly initialized and untrained CNNs, the algorithm discovers models with competent architectures, which after complete training, reach an accuracy of of 92.5% for CIFAR-10 and 94.36% PAMAP2. We further extend the algorithm to include an additional conflicting search objective: the number of parameters of the neural network. Our multi-objective algorithm produces a Pareto optimal set of neural networks, by optimizing the search for both the accuracy and the parameter count, thus emphasizing the versatility of our approach.

show abstract

Section: Related Workmentioning

confidence: 99%

Designing convolutional neural networks with constrained evolutionary piecemeal training

Sapra

Pimentel

2021

Appl Intell

View full text Add to dashboard Cite

show abstract

“…Evolutionary approaches [8,22,27,28,35] use genetic algorithms to optimize the neural architecture. These are population based algorithms where every iteration trains and evaluates all the architectures in the population.…”

Section: Related Workmentioning

confidence: 99%

Constrained Evolutionary Piecemeal Training to Design Convolutional Neural Networks

Sapra

Pimentel

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Neural Architecture Search (NAS), which automates the discovery of efficient neural networks, has demonstrated substantial potential in achieving state of the art performance in a variety of domains such as image classification and language understanding. In most NAS techniques, training of a neural network is considered a separate task or a performance estimation strategy to perform the architecture search. We demonstrate that network architecture and its coefficients can be learned together by unifying concepts of evolutionary search within a population based traditional training process. The consolidation is realised by cleaving the training process into pieces and then put back together in combination with evolution based architecture search operators. We show the competence and versatility of this concept by using datasets from two different domains, CIFAR-10 for image classification and PAMAP2 for human activity recognition. The search is constrained using minimum and maximum bounds on architecture parameters to restrict the size of neural network from becoming too large. Beginning the search from random untrained models, it achieves a fully trained model with a competent architecture, reaching an accuracy of 92.5% and 94.36% on CIFAR-10 and PAMAP2 respectively.

show abstract

“…Some methods use these above blocks directly, such as [9], [36], and some other methods propose other blocks for their own purposes. Chen [107]…”

Section: Constraintsmentioning

confidence: 99%

“…Such computational resources are not available for anyone who is interested in NAS. Retain the best group [26], [36], [38], [42], [49], [55], [73], [77], [84], [97], [111], [114], [132], [156], [157] Discard the worst or the oldest [56], [76], [86], [130], [131], [155], [184] Roulette [29], [30], [44], [60], [62], [107], [113], [116], [133] Tournament selection [9], [19], [20], [31], [59], [63], [80], [93], [99], [108], [117], [120], [125], [127], [128], [184] Others [57], [105] Almost all o...…”

Section: Shorten the Evaluation Timementioning

confidence: 99%

A Survey on Evolutionary Neural Architecture Search

Liu,

Sun,

Xue

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) have achieved great success in many applications, such as image classification, natural language processing and speech recognition. The architectures of DNNs have been proved to play a crucial role in its performance. However, designing architectures for different tasks is a difficult and time-consuming process of trial and error. Neural Architecture Search (NAS), which received great attention in recent years, can design the architecture automatically. Among different kinds of NAS methods, Evolutionary Computation (EC) based NAS methods have recently gained much attention and success. Unfortunately, there is not a comprehensive summary of the EC-based methods. This paper reviews 100+ papers of EC-based NAS methods in light of the common process. Four steps of the process have been covered in this paper including population initialization, population operators, evaluation and selection. Furthermore, current challenges and issues are also discussed to identify future research in this field.

show abstract

Auto-creation of Effective Neural Network Architecture by Evolutionary Algorithm and ResNet for Image Classification

Cited by 20 publications

References 8 publications

Designing convolutional neural networks with constrained evolutionary piecemeal training

Designing convolutional neural networks with constrained evolutionary piecemeal training

Constrained Evolutionary Piecemeal Training to Design Convolutional Neural Networks

A Survey on Evolutionary Neural Architecture Search

Contact Info

Product

Resources

About