Accelerating Neural Architecture Search with Rank-Preserving Surrogate Models

Benmeziane, Hadjer; Ouarnoughi, Hamza; Maghraoui, Kaoutar El; Niar, Smaïl

doi:10.1145/3485557.3485579

Cited by 2 publications

(1 citation statement)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several methods [17] estimate the accuracy. A surrogate model is built to predict the architecture's ranking using the learning to rank theory [3], [18] based on its characteristics. A common practice to assess the search strategies is to compare the number of evaluations required by each algorithm.…”

Section: Background and Related Workmentioning

confidence: 99%

HyT-NAS: Hybrid Transformers Neural Architecture Search for Edge Devices

Mecharbat¹,

Benmeziane²,

Hamza³

et al. 2023

Preprint

View full text Add to dashboard Cite

Vision Transformers have enabled recent attentionbased Deep Learning (DL) architectures to achieve remarkable results in Computer Vision (CV) tasks. However, due to the extensive computational resources required, these architectures are rarely implemented on resource-constrained platforms. Current research investigates hybrid handcrafted convolution-based and attention-based models for CV tasks such as image classification and object detection. In this paper, we propose HyT-NAS, an efficient Hardware-aware Neural Architecture Search (HW-NAS) including hybrid architectures targeting vision tasks on tiny devices. HyT-NAS improves state-of-the-art HW-NAS by enriching the search space and enhancing the search strategy as well as the performance predictors. Our experiments show that HyT-NAS achieves a similar hypervolume with less than 5x training evaluations. Our resulting architecture outperforms MLPerf MobileNetV1 by 6.3% accuracy improvement with 3.5x less number of parameters on Visual Wake Words.

show abstract

Section: Background and Related Workmentioning

confidence: 99%