AnytimeNet: Controlling Time-Quality Tradeoffs in Deep Neural Network Architectures

Kim, Jung-Eun; Bradford, Richard; Shao, Zhong

doi:10.23919/date48585.2020.9116280

Cited by 9 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we first evaluate the impacts of hyper-parameters on the accuracy and latency for predictions using two popular data sets to verify our assertion that hyperparameters affect the inference time and accuracy and analyze which hyper-parameters have more impacts. After that, we evaluate the feasibility and effectiveness of our approach in comparison to layer-wise adaptation, e.g., [24,25]. Since the source code of [24,25] was not available, we have used the depth of a CNN as another hyper-parameter in our model search and evaluation.…”

Section: Evaluation Resultsmentioning

confidence: 99%

“…After that, we evaluate the feasibility and effectiveness of our approach in comparison to layer-wise adaptation, e.g., [24,25]. Since the source code of [24,25] was not available, we have used the depth of a CNN as another hyper-parameter in our model search and evaluation. In this paper, we have used TensorFlow [22] to implement our CNN models.…”

Section: Evaluation Resultsmentioning

confidence: 99%

“…From this, we observe that increasing the total depth of a CNN is a relatively expensive and less cost-effective option. Essentially, real-time image classification by only adapting the number of the layers (e.g., [24,25]) to execute at runtime is substantially more restricted and provides a much narrower scope of adaptation than our approach does. Thus, our approach is more cost-effective.…”

Section: Impacts Of the Total Depthmentioning

confidence: 99%

“…To address the issue, recent works, e.g., [24,25], have investigated how to dynamically skip or add layers to meet timing constraints. Unlike the non-adaptive baseline, we support methodical trade-offs between the inference time and accuracy.…”

Section: Introductionmentioning

confidence: 99%

“…Notably, the layer-wise adaptation method in a single CNN [24,25] has less flexibility for CNN design and run-time adaptation than our approach. When the depth is increased from 8 to 19 in the layer-adaptive baseline, the execution time increases by more than 1.8×, but the accuracy enhances by only 3.6% for CIFAR-10.…”

mentioning

confidence: 98%

See 4 more Smart Citations

Adaptive Deep Learning for Soft Real-Time Image Classification

Chai

Kang

2021

Technologies

View full text Add to dashboard Cite

CNNs (Convolutional Neural Networks) are becoming increasingly important for real-time applications, such as image classification in traffic control, visual surveillance, and smart manufacturing. It is challenging, however, to meet timing constraints of image processing tasks using CNNs due to their complexity. Performing dynamic trade-offs between the inference accuracy and time for image data analysis in CNNs is challenging too, since we observe that more complex CNNs that take longer to run even lead to lower accuracy in many cases by evaluating hundreds of CNN models in terms of time and accuracy using two popular data sets, MNIST and CIFAR-10. To address these challenges, we propose a new approach that (1) generates CNN models and analyzes their average inference time and accuracy for image classification, (2) stores a small subset of the CNNs with monotonic time and accuracy relationships offline, and (3) efficiently selects an effective CNN expected to support the highest possible accuracy among the stored CNNs subject to the remaining time to the deadline at run time. In our extensive evaluation, we verify that the CNNs derived by our approach are more flexible and cost-efficient than two baseline approaches. We verify that our approach can effectively build a compact set of CNNs and efficiently support systematic time vs. accuracy trade-offs, if necessary, to meet the user-specified timing and accuracy requirements. Moreover, the overhead of our approach is little/acceptable in terms of latency and memory consumption.

show abstract

Section: Evaluation Resultsmentioning

confidence: 99%

Section: Evaluation Resultsmentioning

confidence: 99%

Section: Impacts Of the Total Depthmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 98%

See 3 more Smart Citations

Adaptive Deep Learning for Soft Real-Time Image Classification

Chai

Kang

2021

Technologies

View full text Add to dashboard Cite

show abstract

Real-time task scheduling with image resizing for criticality-based machine perception

Liu

Abdelzaher

et al. 2022

Real-Time Syst

View full text Add to dashboard Cite

Scheduling IDK classifiers with arbitrary dependences to minimize the expected time to successful classification

et al. 2023

View full text Add to dashboard Cite

This paper introduces and evaluates a general construct for trading off accuracy and overall execution duration in classification-based machine perception problems—namely, the generalized IDK classifier cascade. The aim is to select the optimal sequence of classifiers required to minimize the expected (i.e. average) execution duration needed to achieve successful classification, subject to a constraint on quality, and optionally a latency constraint on the worst-case execution duration. An IDK classifier is a software component that attempts to categorize each input provided to it into one of a fixed set of classes, returning “I Don’t Know” (IDK) if it is unable to do so with the required level of confidence. An ensemble of several different IDK classifiers may be available for the same classification problem, offering different trade-offs between effectiveness (i.e. the probability of successful classification) and timeliness (i.e. execution duration). A model for representing such characteristics is defined, and a method is proposed for determining the values of the model parameters for a given ensemble of IDK classifiers. Optimal algorithms are developed for sequentially ordering IDK classifiers into an IDK cascade, such that the expected duration to successfully classify an input is minimized, optionally subject to a latency constraint on the worst-case overall execution duration of the IDK cascade. The entire methodology is applied to two real-world case studies. In contrast to prior work, the methodology developed in this paper caters for arbitrary dependences between the probabilities of successful classification for different IDK classifiers. Effective practical solutions are developed considering both single and multiple processors.

show abstract

AnytimeNet: Controlling Time-Quality Tradeoffs in Deep Neural Network Architectures

Cited by 9 publications

References 8 publications

Adaptive Deep Learning for Soft Real-Time Image Classification

Adaptive Deep Learning for Soft Real-Time Image Classification

Real-time task scheduling with image resizing for criticality-based machine perception

Scheduling IDK classifiers with arbitrary dependences to minimize the expected time to successful classification

Contact Info

Product

Resources

About