HSCoNAS: Hardware-Software Co-Design of Efficient DNNs via Neural Architecture Search

Abstract

xiangzho001 1 , shuo001 3 )@e.ntu.edu.sg, (liu.di 2 , liu 4 )@ntu.edu.sg

Systematic Review on Neural Architecture Search

“…There are other papers on the NAS algorithms for IoT and edge devices that we can mention briefly with: different optimization methods [165][166][167][168]; different metrics [169,170]; different Hardware [170][171][172][173][174]; and so on so forth. For finding more information about hardware-aware NAS, you can read [175].…”

Designing Efficient DNNs via Hardware-Aware Neural Architecture Search and Beyond

“…D EEP neural networks (DNNs) have become the de facto engine of artificial intelligence (AI), unlocking unprecedented breakthroughs among a wide range of realworld applications, such as image classification [10], [22], [32], [36], object detection [30], [38], [53], natural language processing (NLP) [40], [43], etc. Subsequently, such breakthroughs provoke an explosion of research interests to deploy state-of-the-art DNNs towards diverse hardware systems to achieve the hardware intelligence [7], [16], [23], [33], [52]. In GoldenNAS, we first search for the optimal architecture candidate for target hardware, and then adapt the searched architecture candidate to satisfy different latency requirements.…”

Hardware-aware neural architecture search and compression towards embedded intelligence

“…MobileNets [6,41] and Shu✏eNets [42,43]. For example, the block-based search space in ProxylessNAS [13] is built on top of MobileNetV2, whereas the block-based search space in HSCoNAS [82] is built on top of Shu✏eNetV2. In parallel, HURRI-CANE [83] demonstrates that di↵erent hardware platforms favor di↵erent search spaces, based on which HURRICANE introduces a hybrid block-based search space that combines both MobileNetV2 and Shu✏eNetV2 to deliver superior architecture solutions.…”

“…To overcome such limitations, several e cient latency prediction strategies have been recently established. For example, [9,13,14,61,82,99] leverage the latency lookup table to approximate the on-device latency for di↵erent architecture candidates. In parallel, [12,24,81,85,92,100,101] turn back to learning-based regression approaches for the latency prediction purpose, which typically train an accurate latency predictor 2.1.…”