MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning

Hsu, Chi-Hung; Chang, Shih-Chieh; Liang, Jhao-Hong; Chou, H.P.; Liu, Chun-Hao; Chang, Shu-Huan; Pan, Tim; Chen, Yuting; Wei, Wei; Juan, Da-Cheng

doi:10.48550/arxiv.1806.10332

Cited by 59 publications

(62 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these methods are usually very slow and require huge resources for training. Other studies [8,12,26] also attempt to optimize multiple objectives like model size and accuracy. Nevertheless, their search process optimizes only on small tasks like CIFAR.…”

Section: Related Workmentioning

confidence: 99%

Lightweight Monocular Depth with a Novel Neural Architecture Search Method

Huynh¹,

Nguyen²,

Matas³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper presents a novel neural architecture search method, called LiDNAS, for generating lightweight monocular depth estimation models. Unlike previous neural architecture search (NAS) approaches, where finding optimized networks is computationally highly demanding, the introduced novel Assisted Tabu Search leads to efficient architecture exploration. Moreover, we construct the search space on a pre-defined backbone network to balance layer diversity and search space size. The LiDNAS method outperforms the state-of-the-art NAS approach, proposed for disparity and depth estimation, in terms of search efficiency and output model performance. The LiDNAS optimized models achieve result superior to compact depth estimation state-of-the-art on NYU-Depth-v2, KITTI, and Scan-Net, while being 7%-500% more compact in size, i.e the number of model parameters.

show abstract

Section: Related Workmentioning

confidence: 99%

Lightweight Monocular Depth with a Novel Neural Architecture Search Method

Huynh¹,

Nguyen²,

Matas³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Perceptible metrics, such as latency and energy consumption, are subject to specification requirements and often more expressive and constrained than accuracy. Therefore, multi-objective optimization became more prominent and has been incorporated into the evaluation strategy [12,22]. However, while acknowledging the importance of perceptible metrics, these NAS designs use proxies like operations after having verified a linear relationship between them [5,32].…”

Section: Related Workmentioning

confidence: 99%

Measuring what Really Matters: Optimizing Neural Networks for TinyML

Heim,

Biri,

et al. 2021

Preprint

View full text Add to dashboard Cite

With the surge of inexpensive computational and memory resources, neural networks (NNs) have experienced an unprecedented growth in architectural and computational complexity. Introducing NNs to resource-constrained devices enables cost-efficient deployments, widespread availability, and the preservation of sensitive data.This work addresses the challenges of bringing Machine Learning to microcontroller units (MCUs), where we focus on the ubiquitous ARM Cortex-M architecture. The detailed effects and trade-offs that optimization methods, software frameworks, and MCU hardware architecture have on key performance metrics such as inference latency and energy consumption have not been previously studied in depth for state-of-the-art frameworks such as TensorFlow Lite Micro. We find that empirical investigations which measure the perceptible metrics -performance as experienced by the userare indispensable, as the impact of specialized instructions and layer types can be subtle. To this end, we propose an implementation-aware design as a cost-effective method for verification and benchmarking. Employing our developed toolchain, we demonstrate how existing NN deployments on resource-constrained devices can be improved by systematically optimizing NNs to their targeted application scenario.

show abstract

“…Hardware-aware NAS: Earlier NAS methods focused on maximizing accuracy under FLOPs constraints [22,25], but low FLOP count does not necessarily translate to hardware efficiency [10,18]. More recent methods incorporate hardware terms (e.g., runtime, power) into cell-based NAS formulations [10,12], but cell-based implementations are not hardware friendly [21]. Breaking away from cell-based assumptions in the search space encoding, recent work employs NAS over a generalized MobileNetV2-based design space introduced in [20].…”

Section: Related Workmentioning

confidence: 99%

Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours

Stamoulis¹,

Ding²,

Wang³

et al. 2019

Preprint

View full text Add to dashboard Cite

Can we automatically design a Convolutional Network (Con-vNet) with the highest image classification accuracy under the latency constraint of a mobile device? Neural architecture search (NAS) has revolutionized the design of hardware-efficient ConvNets by automating this process. However, the NAS problem remains challenging due to the combinatorially large design space, causing a significant searching time (at least 200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a novel differentiable NAS method for designing hardware-efficient ConvNets in less than 4 hours. Our contributions are as follows: 1. Single-path search space: Compared to previous differentiable NAS methods, Single-Path NAS uses one singlepath over-parameterized ConvNet to encode all architectural decisions with shared convolutional kernel parameters, hence drastically decreasing the number of trainable parameters and the search cost down to few epochs. 2. Hardware-efficient ImageNet classification: Single-Path NAS achieves 74.96% top-1 accuracy on ImageNet with 79ms latency on a Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with similar inference latency constraints (≤ 80ms). 3. NAS efficiency: Single-Path NAS search cost is only 8 epochs (30 TPUhours), which is up to 5,000× faster compared to prior work. 4. Reproducibility: Unlike all recent mobile-efficient NAS methods which only release pretrained models, we open-source our entire codebase at: https://github.com/dstamoulis/single-path-nas.

show abstract

MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning

Cited by 59 publications

References 7 publications

Lightweight Monocular Depth with a Novel Neural Architecture Search Method

Lightweight Monocular Depth with a Novel Neural Architecture Search Method

Measuring what Really Matters: Optimizing Neural Networks for TinyML

Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours

Contact Info

Product

Resources

About