NSGA-Net

Lu, Zhichao; Whalen, Ian; Boddeti, Vishnu Naresh; Dhebar, Yashesh; Deb, Kalyanmoy; Goodman, Erik D.; Banzhaf, Wolfgang

doi:10.1145/3321707.3321729

Cited by 339 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hu et al (2021) proposed a new performance estimation metric named random-weight evaluation (RWE) to quantify the quality of CNNs. Lu et al (2018) proposed NSGANet, an evolutionary algorithm that combines prior knowledge from handcrafted architectures with an exploration comprising crossover and mutation. Some software packages provide search functions, such as pyGPGO and Optunity (Bergstra et al, 2011), Hyperopt-Sklearn (Bergstra et al, 2015), etc.…”

Section: (C) Other Improved Nas Methodsmentioning

confidence: 99%

“…Most neural networks can be divided into three types: (a) professional knowledge is required for manual design, such as VGG (Ferentinos, 2018) and ResNet (He et al, 2016); (b) semiautomatic design methods like genetic neural networks (Xie and Yuille, 2017), hierarchical evolution , and others; and (c) fully automatic design, such as when Google introduces the neural architecture search (NAS) concept (Zoph and Le, 2016), which has received considerable attention (Baker et al, 2016;Lu et al, 2018). NAS can search for the best hyperparameters to perform better than manual design.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A novel heuristic target-dependent neural architecture search method with small samples

Rao

et al. 2022

Front. Plant Sci.

View full text Add to dashboard Cite

It is well known that crop classification is essential for genetic resources and phenotype development. Compared with traditional methods, convolutional neural networks can be utilized to identify features automatically. Nevertheless, crops and scenarios are quite complex, which makes it challenging to develop a universal classification method. Furthermore, manual design demands professional knowledge and is time-consuming and labor-intensive. In contrast, auto-search can create network architectures when faced with new species. Using rapeseed images for experiments, we collected eight types to build datasets (rapeseed dataset (RSDS)). In addition, we proposed a novel target-dependent search method based on VGGNet (target-dependent neural architecture search (TD-NAS)). The result shows that test accuracy does not differ significantly between small and large samples. Therefore, the influence of the dataset size on generalization is limited. Moreover, we used two additional open datasets (Pl@ntNet and ICL-Leaf) to test and prove the effectiveness of our method due to three notable features: (a) small sample sizes, (b) stable generalization, and (c) free of unpromising detections.

show abstract

Section: (C) Other Improved Nas Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A novel heuristic target-dependent neural architecture search method with small samples

Rao

et al. 2022

Front. Plant Sci.

View full text Add to dashboard Cite

show abstract

“… Our data is relevant for researchers developing tools for NN design. Such tools include neural architecture search [6] , [7] , [8] and methods for NN fitness prediction and training termination [9] , [10] , [11] . Learning curve data is essential to the development of methods for NN fitness modeling and prediction [3] , [12] .…”

Section: Specifications Tablementioning

confidence: 99%

High frequency accuracy and loss data of random neural networks trained on image datasets

et al. 2022

View full text Add to dashboard Cite

Neural Networks (NNs) are increasingly used across scientific domains to extract knowledge from experimental or computational data. An NN is composed of natural or artificial neurons that serve as simple processing units and are interconnected into a model architecture; it acquires knowledge from the environment through a learning process and stores this knowledge in its connections. The learning process is conducted by training. During NN training, the learning process can be tracked by periodically validating the NN and calculating its fitness. The resulting sequence of fitness values (i.e., validation accuracy or validation loss) is called the NN learning curve. The development of tools for NN design requires knowledge of diverse NNs and their complete learning curves. Generally, only final fully-trained fitness values for highly accurate NNs are made available to the community, hampering efforts to develop tools for NN design and leaving unaddressed aspects such as explaining the generation of an NN and reproducing its learning process. Our dataset fills this gap by fully recording the structure, metadata, and complete learning curves for a wide variety of random NNs throughout their training. Our dataset captures the lifespan of 6000 NNs throughout generation, training, and validation stages. It consists of a suite of 6000 tables, each table representing the lifespan of one NN. We generate each NN with randomized parameter values and train it for 40 epochs on one of three diverse image datasets (i.e., CIFAR-100, FashionMNIST, SVHN). We calculate and record each NN’s fitness with high frequency—every half epoch—to capture the evolution of the training and validation process. As a result, for each NN, we record the generated parameter values describing the structure of that NN, the image dataset on which the NN trained, and all loss and accuracy values for the NN every half epoch. We put our dataset to the service of researchers studying NN performance and its evolution throughout training and validation. Statistical methods can be applied to our dataset to analyze the shape of learning curves in diverse NNs, and the relationship between an NN’s structure and its fitness. Additionally, the structural data and metadata that we record enable the reconstruction and reproducibility of the associated NN.

show abstract

“…[23] 96.25 15.7 300 EA CNN-GA [38] 96.78 2.9 35 EA CGP-CNN [35] 94.02 1.7 27 EA AE-CNN [37] 95.7 2.0 27 EA NSGANetV1-A2 [27] 97.35 0.9 27 EA AE-CNN+E2EPP [36] 94.70 4.3 7 EA NSGA-NET [26] 97.25 [14] 77.90 1.7 -manual DenseNet-BC [15] 82.82 25.6 -manual ShuffleNet [43] 77.14 1.06 -manual PNAS [22] 80.47 3.2 225 SMBO MetaQNN [1] 72.86 11.2 90 RL ENAS [29] 80.57 4.6 0.45 RL AmoebaNet-A [30] 81.07 3.2 3150 EA Large-scale Evo. [31] 77.00 40.4 2750 EA CNN-GA [38] 79.47 4.1 40 EA AE-CNN [37] 79.15 5.4 36 EA NSGANetV1-A2 [27] 82.58 0.9 27 EA Genetic CNN [39] 70.95 -17 EA AE-CNN+E2EPP [36] 77.98 20.9 10 EA NSGA-NET [26] 79. ment for the selection method. The nearest-neighbors size for the novelty search is 5.…”

Section: Architecturementioning

confidence: 99%

Novelty Driven Evolutionary Neural Architecture Search

Sinha¹,

Chen²

2022

Preprint

View full text Add to dashboard Cite

Evolutionary algorithms (EA) based neural architecture search (NAS) involves evaluating each architecture by training it from scratch, which is extremely time-consuming. This can be reduced by using a supernet for estimating the fitness of an architecture due to weight sharing among all architectures in the search space. However, the estimated fitness is very noisy due to the co-adaptation of the operations in the supernet which results in NAS methods getting trapped in local optimum. In this paper, we propose a method called NEvoNAS wherein the NAS problem is posed as a multiobjective problem with 2 objectives: (i) maximize architecture novelty, (ii) maximize architecture fitness/accuracy. The novelty search is used for maintaining a diverse set of solutions at each generation which helps avoiding local optimum traps while the architecture fitness is calculated using supernet. NSGA-II is used for finding the pareto optimal front for the NAS problem and the best architecture in the pareto front is returned as the searched architecture. Exerimentally, NEvoNAS gives better results on 2 different search spaces while using significantly less computational resources as compared to previous EA-based methods. The code for our paper can be found here.

show abstract

NSGA-Net

Cited by 339 publications

References 14 publications

A novel heuristic target-dependent neural architecture search method with small samples

A novel heuristic target-dependent neural architecture search method with small samples

High frequency accuracy and loss data of random neural networks trained on image datasets

Novelty Driven Evolutionary Neural Architecture Search

Contact Info

Product

Resources

About