NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm (Extended Abstract)

Abstract

<jats:p>Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and elaborate design. Recently, neural architecture search (NAS) was proposed with the aim of automating the network design process and generating task-dependent architectures. This paper introduces NSGA-Net -- an evolutionary search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures, and finally an exploitation step that utilizes the hidden useful knowledge stored in the entire history of evaluated neural architectures in the form of a Bayesian Network. The integration of these components allows an efficient design of architectures that are competitive and in many cases outperform both manually and automatically designed architectures on CIFAR-10 classification task. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.</jats:p>

Defect Detection of Industry Wood Veneer Based on NAS and Multi-Channel Mask R-CNN

“…To solve the two-level optimization problem, a genetic algorithm was used for the optimization of and Stochastic Gradient Descent–Momentum (SGD-M) was used for the optimization of . The genetic algorithm has been widely used for NAS, and it has high robustness in many applications [ 42 , 43 , 44 , 45 ]. By using the genetic algorithm, the original traversing each structure to select the best problem was transformed into optimizing in a large search space, which efficiently traversed the space.…”

“…where w . The genetic algorithm has been widely used for NAS, and it has high robustness in many applications [42][43][44][45]. By using the genetic algorithm, the original traversing each structure to select the best problem was transformed into optimizing in a large search space, which efficiently traversed the space.…”

From federated learning to federated neural architecture search: a survey

“…Rather than generating the entire CNNs, the micro-search space [46] has also been successfully employed by many recent EA-based NAS algorithms [83][84][85][86][87]. Real et al [85] propose an extension of the large-scale evolution [73], called AmoebaNet, which has achieved better results on ImageNet compared with hand-designed methods for the first time.…”

“…One of the earliest evolutionary multi-objective methods to design CNNs is NEMO [96], which simultaneously optimizes classification performance and inference time of a network based on NSGA-II [97]. Inspired by NEMO, Lu et al [84] consider classification error and computational complexity as the two objectives. In addition, they empirically test multiple computational complexity metrics to measure the inference time of a network containing the number of active layers, the number of activating connections between layers, and the number of floating-point operations (FLOPs).…”

“…Moreover, a Bayesian network (BN) is adopted to learn the knowledge about promising architectures present in the search history and then guide the future exploitation in generating new architectures. Subsequently, Lu et al suggest NSGANet-v2 [87], an extension of NSGANet [84], where a comprehensive search space including more layer operations and one more option that controls the width of the model is introduced. Dong et al [98] present a DPP-Net on the basis of [99] that optimizes both GPU memory usage and the model performance.…”