DynExit: A Dynamic Early-Exit Strategy for Deep Residual Networks

Wang, Meiqi; Mo, Jianqiao; Lin, Jun; Wang, Zhongfeng; Du, Li

doi:10.1109/sips47522.2019.9020551

Cited by 28 publications

(14 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As we discussed, in a perfectly calibrated network, this criterion matches exactly the accuracy [10]. Early-exit DNNs are also considered in [11], which uses a residual neural network (ResNet) implemented on specific hardware. Other works propose an optimization problem to select the most suitable DNN depth for model partitioning.…”

Section: Related Workmentioning

confidence: 94%

Calibration-Aided Edge Inference Offloading via Adaptive Model Partitioning of Deep Neural Networks

Pacheco¹,

Couto²,

Simeone³

2020

Preprint

View full text Add to dashboard Cite

Mobile devices can offload deep neural network (DNN)-based inference to the cloud, overcoming local hardware and energy limitations. However, offloading adds communication delay, thus increasing the overall inference time, and hence it should be used only when needed. An approach to address this problem consists of the use of adaptive model partitioning based on early-exit DNNs. Accordingly, the inference starts at the mobile device, and an intermediate layer estimates the accuracy: If the estimated accuracy is sufficient, the device takes the inference decision; Otherwise, the remaining layers of the DNN run at the cloud. Thus, the device offloads the inference to the cloud only if it cannot classify a sample with high confidence. This offloading requires a correct accuracy prediction at the device. Nevertheless, DNNs are typically miscalibrated, providing overconfident decisions. This work shows that the employment of a miscalibrated early-exit DNN for offloading via model partitioning can significantly decrease inference accuracy. In contrast, we argue that implementing a calibration algorithm prior to deployment can solve this problem, allowing for more reliable offloading decisions.

show abstract

Section: Related Workmentioning

confidence: 94%

Calibration-Aided Edge Inference Offloading via Adaptive Model Partitioning of Deep Neural Networks

Pacheco¹,

Couto²,

Simeone³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The first approach is used to determine whether the model input is transferred to the edge server, and the latter dynamically adjusts the number of layers to be used as an auxiliary neural model deployed on mobile device for efficient usage of communication channels in EC systems. Neshatpour et al [78] decomposes a LeNet-5 [55] AlexNet [50] ResNet [32] Link Teerapittayanon et al [110] Image classification* Multi-camera multi-object detection [92] Distributed DNNs Link Lo et al [68] Image classification CIFAR-10/100 [49] NiN [63] ResNet [32] WRN [131] Neshatpour et al [78] Image classification ImageNet [93] AlexNet [50] Zeng et al [132] Image classification CIFAR-10 [49] AlexNet [50] Wang et al [115] Image classification CIFAR-10/100 [49] ResNet [32] Li et al [61] Image classification CIFAR-10/100 [49] ImageNet (2012) [93] MSDNet [37] Link…”

Section: Ee For CV Applicationsmentioning

confidence: 99%

Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges

Matsubara¹,

Levorato²,

Restuccia³

2021

Preprint

View full text Add to dashboard Cite

Mobile devices such as smartphones and autonomous vehicles increasingly rely on deep neural networks (DNNs) to execute complex inference tasks such as image classification and speech recognition, among others. However, continuously executing the entire DNN on the mobile device can quickly deplete its battery. Although task offloading to edge devices may decrease the mobile device's computational burden, erratic patterns in channel quality, network and edge server load can lead to a significant delay in task execution. Recently, approaches based on split computing (SC) have been proposed, where the DNN is split into a head and a tail model, executed respectively on the mobile device and on the edge device. Ultimately, this may reduce bandwidth usage as well as energy consumption. Another approach, called early exiting (EE), trains models to present multiple "exits" earlier in the architecture, each providing increasingly higher target accuracy. Therefore, the trade-off between accuracy and delay can be tuned according to the current conditions or application demands. In this paper, we provide a comprehensive survey of the state of the art in SC and EE strategies, by presenting a comparison of the most relevant approaches. We conclude the paper by providing a set of compelling research challenges. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing; • Computer systems organization → Embedded and cyber-physical systems; • Computing methodologies → Neural networks.

show abstract

“…Instead of searching for the optimal network configuration for fixed hardware, another set of approaches is to design the hardware specifically for early-exit networks [13,36,37] or co-design the network and hardware for efficient progressive inference [57,73].…”

Section: Early-exits and Target Hardwarementioning

confidence: 99%

“…Paul et al [57] Vision/Classification Efficient EE inference on FPGA. DynExit [73] Vision/Classification Trainable weights in joint EE loss and FPGA dployment.…”

Section: Ee-h/w (Co-)designmentioning

confidence: 99%

Adaptive Inference through Early-Exit Networks

Laskaridis

Kouris

Lane

2021

Proceedings of the 5th International Workshop on Embedded and Mobile Deep Learning

View full text Add to dashboard Cite

DNNs are becoming less and less over-parametrised due to recent advances in efficient model design, through careful hand-crafted or NAS-based methods. Relying on the fact that not all inputs require the same amount of computation to yield a confident prediction, adaptive inference is gaining attention as a prominent approach for pushing the limits of efficient deployment. Particularly, early-exit networks comprise an emerging direction for tailoring the computation depth of each input sample at runtime, offering complementary performance gains to other efficiency optimisations. In this paper, we decompose the design methodology of early-exit networks to its key components and survey the recent advances in each one of them. We also position early-exiting against other efficient inference solutions and provide our insights on the current challenges and most promising future directions for research in the field. CCS CONCEPTS• Computing methodologies → Neural networks; • Humancentered computing → Ubiquitous and mobile computing systems and tools.

show abstract

DynExit: A Dynamic Early-Exit Strategy for Deep Residual Networks

Cited by 28 publications

References 14 publications

Calibration-Aided Edge Inference Offloading via Adaptive Model Partitioning of Deep Neural Networks

Calibration-Aided Edge Inference Offloading via Adaptive Model Partitioning of Deep Neural Networks

Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges

Adaptive Inference through Early-Exit Networks

Contact Info

Product

Resources

About