MSNet: Structural Wired Neural Architecture Search for Internet of Things

Cheng, Hsin-Pai; Zhang, Tunhou; Yang, Yukun; Yan, Feng; Teague, Harris; Chen, Yiran; Li, Hai

doi:10.1109/iccvw.2019.00254

Cited by 16 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Measuring the actual latency for each candidate architecture takes about 20 seconds or more (to average out the random variations as per TensorFlow-Lite guideline [22] and also suggested by [10]). Meanwhile, the total number of candidate architectures sampled by a NAS algorithm is typically in the order of 10k or even more [12,15,40], thus settling the total latency evaluation time to be 50+ hours for just one target device.…”

Section: Current Practice For Reducing the Cost Of Performance Evalua...mentioning

confidence: 99%

One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search

Jiang

Shi

et al. 2021

Proc. ACM Meas. Anal. Comput. Syst.

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) are used in numerous real-world applications such as vision-based autonomous driving and video content analysis. To run CNN inference on various target devices, hardware-aware neural architecture search (NAS) is crucial. A key requirement of efficient hardware-aware NAS is the fast evaluation of inference latencies in order to rank different architectures. While building a latency predictor for each target device has been commonly used in state of the art, this is a very time-consuming process, lacking scalability in the presence of extremely diverse devices. In this work, we address the scalability challenge by exploiting latency monotonicity --- the architecture latency rankings on different devices are often correlated. When strong latency monotonicity exists, we can re-use architectures searched for one proxy device on new target devices, without losing optimality. In the absence of strong latency monotonicity, we propose an efficient proxy adaptation technique to significantly boost the latency monotonicity. Finally, we validate our approach and conduct experiments with devices of different platforms on multiple mainstream search spaces, including MobileNet-V2, MobileNet-V3, NAS-Bench-201, ProxylessNAS and FBNet. Our results highlight that, by using just one proxy device, we can find almost the same Pareto-optimal architectures as the existing per-device NAS, while avoiding the prohibitive cost of building a latency predictor for each device.

show abstract

Section: Current Practice For Reducing the Cost Of Performance Evalua...mentioning

confidence: 99%

One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search

Jiang

Shi

et al. 2021

Proc. ACM Meas. Anal. Comput. Syst.

View full text Add to dashboard Cite

show abstract

“…Figure 8 compares our DNAS results (MicroNets) to three state of the art results, including ProxylessNAS (Cai et al, 2019), MSNet (Cheng et al, 2019), and the TFLM example model (Chowdhery et al, 2019). The largest network in our search space is MobileNetV2, which achieves 88.75% accuracy.…”

Section: Visual Wake Words (Vww)mentioning

confidence: 97%

MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers

Banbury,

Zhou,

Fedorov

et al. 2020

Preprint

View full text Add to dashboard Cite

Executing machine learning workloads locally on resource constrained microcontrollers (MCUs) promises to drastically expand the application space of IoT. However, so-called TinyML presents severe technical challenges, as deep neural network inference demands a large compute and memory budget. To address this challenge, neural architecture search (NAS) promises to help design accurate ML models that meet the tight MCU memory, latency and energy constraints. A key component of NAS algorithms is their latency/energy model, i.e., the mapping from a given neural network architecture to its inference latency/energy on an MCU. In this paper, we observe an intriguing property of NAS search spaces for MCU model design: on average, model latency varies linearly with model operation (op) count under a uniform prior over models in the search space. Exploiting this insight, we employ differentiable NAS (DNAS) to search for models with low memory usage and low op count, where op count is treated as a viable proxy to latency. Experimental results validate our methodology, yielding our MicroNet models, which we deploy on MCUs using Tensorflow Lite Micro, a standard open-source NN inference runtime widely used in the TinyML community. MicroNets demonstrate state-of-the-art results for all three TinyMLperf industry-standard benchmark tasks: visual wake words, audio keyword spotting, and anomaly detection.

show abstract

“…RouteNet is used to predict delay in a network topology. In [ 34 ], an IoT network was used to construct DNNs to perform ML. In [ 35 ], the focus was on the creation of a graph recovery model using a gated GCN, which sends information from available sensors to missing sensors in order to reconstruct their features.…”

Section: Related Workmentioning

confidence: 99%

A Survey of AI-Based Anomaly Detection in IoT and Sensor Networks

DeMedeiros

Hendawi

Álvarez

2023

Sensors

View full text Add to dashboard Cite

Machine learning (ML) and deep learning (DL), in particular, are common tools for anomaly detection (AD). With the rapid increase in the number of Internet-connected devices, the growing desire for Internet of Things (IoT) devices in the home, on our person, and in our vehicles, and the transition to smart infrastructure and the Industrial IoT (IIoT), anomaly detection in these devices is critical. This paper is a survey of anomaly detection in sensor networks/the IoT. This paper defines what an anomaly is and surveys multiple sources based on those definitions. The goal of this survey was to highlight how anomaly detection is being performed on the Internet of Things and sensor networks, identify anomaly detection approaches, and outlines gaps in the research in this domain.

show abstract

MSNet: Structural Wired Neural Architecture Search for Internet of Things

Cited by 16 publications

References 4 publications

One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search

One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search

MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers

A Survey of AI-Based Anomaly Detection in IoT and Sensor Networks

Contact Info

Product

Resources

About