Learning Layer-Skippable Inference Network

Jiang, Yu‐Gang; Cheng, Changmao; Lin, Hangyu; Fu, Yanwei

doi:10.1109/tip.2020.3018269

Cited by 5 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Object detection (face [38], [181], [182], facial point [183], pedestrian [184], general [31], [185], [186], [187], [188]) Image segmentation [101], [189], Super resolution [190], Style transfer [191], Coarse-to-fine classification [192] I & S…”

Section: Tablementioning

confidence: 99%

Dynamic Neural Networks: A Survey

Han

Huang

Song

et al. 2021

Preprint

View full text Add to dashboard Cite

Dynamic neural network is an emerging research topic in deep learning. Compared to static models which have fixed computational graphs and parameters at the inference stage, dynamic networks can adapt their structures or parameters to different inputs, leading to notable advantages in terms of accuracy, computational efficiency, adaptiveness, etc. In this survey, we comprehensively review this rapidly developing area by dividing dynamic networks into three main categories: 1) instance-wise dynamic models that process each instance with data-dependent architectures or parameters; 2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts. The important research problems of dynamic networks, e.g., architecture design, decision making scheme, optimization technique and applications, are reviewed systematically. Finally, we discuss the open problems in this field together with interesting future research directions.

show abstract

Section: Tablementioning

confidence: 99%

Dynamic Neural Networks: A Survey

Han

Huang

Song

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Similarly, in [17,34,45,53], smaller models are obtained using a subset of the channels (feature maps) of the largest one, whereas in [26] they are obtained varying the quantization bit-width. Other advanced adaptive inference mechanisms are described in [12,27,28]. Importantly, many of these papers have shown that, if "easy" inputs are the majority at test time, input-adaptive systems can not only enable a higher flexibility in terms of operating modes, but also reach better trade-offs with respect to using multiple independent static networks.…”

Section: Flexibility-oriented Optimization With Input-adaptive Variab...mentioning

confidence: 99%

Human Activity Recognition on Microcontrollers with Quantized and Adaptive Deep Neural Networks

Daghero

Burrello

Xie

et al. 2022

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

Human Activity Recognition (HAR) based on inertial data is an increasingly diffused task on embedded devices, from smartphones to ultra low-power sensors. Due to the high computational complexity of deep learning models, most embedded HAR systems are based on simple and not-so-accurate classic machine learning algorithms. This work bridges the gap between on-device HAR and deep learning, proposing a set of efficient one-dimensional Convolutional Neural Networks (CNNs) that can be deployed on general purpose microcontrollers (MCUs). Our CNNs are obtained combining hyper-parameters optimization with sub-byte and mixed-precision quantization, to find good trade-offs between classification results and memory occupation. Moreover, we also leverage adaptive inference as an orthogonal optimization to tune the inference complexity at runtime based on the processed input, hence producing a more flexible HAR system. With experiments on four datasets, and targeting an ultra-low-power RISC-V MCU, we show that (i) we are able to obtain a rich set of Pareto-optimal CNNs for HAR, spanning more than 1 order of magnitude in terms of memory, latency, and energy consumption; (ii) thanks to adaptive inference, we can derive >20 runtime operating modes starting from a single CNN, differing by up to 10% in classification scores and by more than 3× in inference complexity, with a limited memory overhead; (iii) on three of the four benchmarks, we outperform all previous deep learning methods, while reducing the memory occupation by more than 100×. The few methods that obtain better performance (both shallow and deep) are not compatible with MCU deployment; (iv) all our CNNs are compatible with real-time on-device HAR, achieving an inference latency that ranges between 9 μs and 16 ms. Their memory occupation varies in 0.05–23.17 kB, and their energy consumption in 0.05 and 61.59 μJ, allowing years of continuous operation on a small battery supply.

show abstract