<i>LightNAS</i>: On Lightweight and Scalable Neural Architecture Search for Embedded Platforms

Luo, Xiangzhong; Liu, Di; Kong, Hao; Huai, Shuo; Chen, Hui; Liu, Weichen

doi:10.1109/tcad.2022.3208187

Cited by 14 publications

(22 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, [9], as the first attempt, empirically accumulates the losses collected from the early training epochs, which delivers promising correlation performance on NAS-Bench-201 [2]. Nonetheless, as pointed out in [7], [9] ignores the training dynamics of different architectures, especially in the early training phase, which leads to unstable and even biased estimation. To alleviate this issue, [7] proposes to measure and accumulate the batchwise loss descent, thereby achieving better correlation performance than [9] under the same training budgets.…”

Section: Trained Batchwise Estimationmentioning

confidence: 99%

“…In the field of NAS, it is critical to reliably estimate the performance (i.e., accuracy) of possible architectures. Nonetheless, traditional approaches prefer to train possible architectures from scratch for a large number of epochs, which indeed provides an accurate estimation but suffers from prohibitive computational overheads [7,9]. To this end, we focus on the following question:…”

Section: Introductionmentioning

confidence: 99%

“…In particular, TBE is able to reliably reflect the accuracy of possible architectures using the batchwise training statistics collected from the very early epochs. Besides, different from previous relevant methods [7,9], TBE only requires part of the training dataset. Specifically, the main insight behind TBE is an accuracy predictor, which is trained using the early batchwise training statistics.…”

Section: Introductionmentioning

confidence: 99%

“…Specifically, the main insight behind TBE is an accuracy predictor, which is trained using the early batchwise training statistics. Extensive experiments on NAS-Bench-201 [2] clearly show that TBE achieves better correlation performance than previous relevant proxies [7,9] with even less training budgets. We then leverage a popular query-based NAS algorithm [5] as the search engine to examine TBE in the hardware-aware NAS scenario, leading to promising improvements in terms of both accuracy on target task [2] and latency on target hardware [4].…”

Section: Introductionmentioning

confidence: 99%

“…This work is supported by Nanyang Technological University, Singapore, under its NAP (M4082282) and SUG (M4082087). [2] using TBE, BTE [7], and TSE [9], respectively.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Work-in-Progress: What to Expect of Early Training Statistics? An Investigation on Hardware-Aware Neural Architecture Search

Luo

Liu

Kong

et al. 2022

2022 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS)

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Trained Batchwise Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…This work is supported by Nanyang Technological University, Singapore, under its NAP (M4082282) and SUG (M4082087). [2] using TBE, BTE [7], and TSE [9], respectively.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Work-in-Progress: What to Expect of Early Training Statistics? An Investigation on Hardware-Aware Neural Architecture Search

Luo

Liu

Kong

et al. 2022

2022 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS)

Self Cite

View full text Add to dashboard Cite

show abstract

Modular design automation of the morphologies, controllers, and vision systems for intelligent robots: a survey

Wang

Mai

et al. 2023

Vis. Intell.

View full text Add to dashboard Cite

Design automation is a core technology in industrial design software and an important branch of knowledge-worker automation. For example, electronic design automation (EDA) has played an important role in both academia and industry. Design automation for intelligent robots refers to the construction of unified modular graph models for the morphologies (body), controllers (brain), and vision systems (eye) of intelligent robots under digital twin architectures, which effectively supports the automation of the morphology, controller, and vision system design processes of intelligent robots by taking advantage of the powerful capabilities of genetic programming, evolutionary computation, deep learning, reinforcement learning, and causal reasoning in model representation, optimization, perception, decision making, and reasoning. Compared with traditional design methods, MOdular DEsigN Automation (MODENA) methods can significantly improve the design efficiency and performance of robots, effectively avoiding the repetitive trial-and-error processes of traditional design methods, and promoting automatic discovery of innovative designs. Thus, it is of considerable research significance to study MODENA methods for intelligent robots. To this end, this paper provides a systematic and comprehensive overview of applying MODENA in intelligent robots, analyzes the current problems and challenges in the field, and provides an outlook for future research. First, the design automation for the robot morphologies and controllers is reviewed, individually, with automated design of control strategies for swarm robots also discussed, which has emerged as a prominent research focus recently. Next, the integrated design automation of both the morphologies and controllers for robotic systems is presented. Then, the design automation of the vision systems of intelligent robots is summarized when vision systems have become one of the most important modules for intelligent robotic systems. Then, the future research trends of integrated “Body-Brain-Eye” design automation for intelligent robots are discussed. Finally, the common key technologies, research challenges and opportunities in MODENA for intelligent robots are summarized.

show abstract

OnceNAS: Discovering efficient on-device inference neural networks for edge devices

Zhang,

Qin,

Zhang

et al. 2024

Information Sciences

View full text Add to dashboard Cite

LightNAS: On Lightweight and Scalable Neural Architecture Search for Embedded Platforms

Cited by 14 publications

References 21 publications

Work-in-Progress: What to Expect of Early Training Statistics? An Investigation on Hardware-Aware Neural Architecture Search

Work-in-Progress: What to Expect of Early Training Statistics? An Investigation on Hardware-Aware Neural Architecture Search

Modular design automation of the morphologies, controllers, and vision systems for intelligent robots: a survey

OnceNAS: Discovering efficient on-device inference neural networks for edge devices

Contact Info

Product

Resources

About