ReNAS: Relativistic Evaluation of Neural Architecture Search

Xu, Yixing; Wang, Yunhe; Han, Kai; Jui, Shangling; Xu, Chunjing; Xu, Chang

doi:10.1109/cvpr46437.2021.00439

Cited by 60 publications

(56 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are five representative state-of-the-art neural predictors: Peephole [7], E2EPP [28], Semi-Supervised Assessor of Neural Architectures (SSANA) [33], Neural Predictor for Neural Architecture Search (NPNAS) [34] and ReNAS [37]. All of them have been investigated on image classification tasks owning to various well-designed architectures and available benchmark datasets for classifying images.…”

Section: Neural Predictormentioning

confidence: 99%

“…The traditional hard encoding [37] uses an integer vector to encode layer type x t n directly. Then, the type vector is broadcasted into the adjacency matrix x m n which can be expressed in Equation 2:…”

Section: Architecture Encodingmentioning

confidence: 99%

“…However, the layers close to the In layer are always placed in front of the layer type list, and the layers close to the Out layer can only be placed at the back. In addition, null layers are generally placed at the end [34,37]. In this way, the layers that should have no order and position are sorted in a breadth-first traversal manner.…”

Section: Homogeneous Architecture Augmentationmentioning

confidence: 99%

“…Because of the character of the regression model, the encoded architecture information must be fixed length. We refer the padding approach in [37] to achieve this goal, by adding zeros into the adjacency matrix and the layer type list. The position of starting to pad zeros is the penultimate element in the layer type list, while the starting positions in the adjacency matrix are the penultimate row and column.…”

Section: Homogeneous Architecture Augmentationmentioning

confidence: 99%

“…The proposed HAAP algorithm is compared with the state-of-the-art algorithms which we have introduced above, i.e., Peephole [7], E2EPP [28], SSANA [33], ReNAS [37] and NPNAS [34]. NAS-Bench-101 is used to providing training and testing architectures.…”

Section: Comparison Of Prediction Performance On Nas-bench-101mentioning

confidence: 99%

See 4 more Smart Citations

Homogeneous Architecture Augmentation for Neural Predictor

Liu

Sun

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Neural Architecture Search (NAS) can automatically design well-performed architectures of Deep Neural Networks (DNNs) for the tasks at hand. However, one bottleneck of NAS is the prohibitively computational cost largely due to the expensive performance evaluation. The neural predictors can directly estimate the performance without any training of the DNNs to be evaluated, thus have drawn increasing attention from researchers. Despite their popularity, they also suffer a severe limitation: the shortage of annotated DNN architectures for effectively training the neural predictors. In this paper, we proposed Homogeneous Architecture Augmentation for Neural Predictor (HAAP) of DNN architectures to address the issue aforementioned. Specifically, a homogeneous architecture augmentation algorithm is proposed in HAAP to generate sufficient training data taking the use of homogeneous representation. Furthermore, the one-hot encoding strategy is introduced into HAAP to make the representation of DNN architectures more effective. The experiments have been conducted on both NAS-Benchmark-101 and NAS-Bench-201 dataset. The experimental results demonstrate that the proposed HAAP algorithm outperforms the state of the arts compared, yet with much less training data. In addition, the ablation studies on both benchmark datasets have also shown the universality of the homogeneous architecture augmentation. Our code has been made available at https://github.com/lyq998/HAAP.

show abstract

Section: Neural Predictormentioning

confidence: 99%

Section: Architecture Encodingmentioning

confidence: 99%

Section: Homogeneous Architecture Augmentationmentioning

confidence: 99%

Section: Homogeneous Architecture Augmentationmentioning

confidence: 99%

Section: Comparison Of Prediction Performance On Nas-bench-101mentioning

confidence: 99%

See 3 more Smart Citations

Homogeneous Architecture Augmentation for Neural Predictor

Liu

Sun

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Ranking‐based architecture generation for surrogate‐assisted neural architecture search

Xiao,

Wang

2024

Concurrency and Computation

View full text Add to dashboard Cite

Architectures generation optimization has been received a lot of attention in neural architecture search (NAS) since its efficiency in generating architecture. By learning the architecture representation through unsupervised learning and constructing a latent space, the prediction process of predictors is simplified, leading to improved efficiency in architecture search. However, searching for architectures with top performance in complex and large NAS search spaces remains challenging. In this paper, an approach that combined a ranker and generative model is proposed to address this challenge through regularizing the latent space and identifying architectures with top rankings. We introduce the ranking error to gradually regulate the training of the generative model, making it easier to identify architecture representations in the latent space. Additionally, a surrogate‐assisted evolutionary search method that utilized neural network Bayesian optimization is proposed to efficiently explore promising architectures in the latent space. We demonstrate the benefits of our approach in optimizing architectures with top rankings, and our method outperforms state‐of‐the‐art techniques on various NAS benchmarks. The code is available at https://github.com/outofstyle/RAGS‐NAS.

show abstract