Design and Optimization of Energy-Accuracy Tradeoff Networks for Mobile Platforms via Pretrained Deep Models

Jayakodi, Nitthilan Kanappan; Belakaria, Syrine; Deshwal, Aryan; Doppa, Janardhan Rao

doi:10.1145/3366636

Cited by 23 publications

(14 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While DNN models can be deployed on these inteligent edge platforms by specific runtime systems, which are usually closed-source or unmodifiable, the model compression techniques can be used to further optimize the inference performance. Besides, there are several studies on the adaptive inference for optimizing deep learning on embedded platforms, including adaptive strategies for neural network inference [22]- [25] and hardware/software co-design [26]- [28], which allow deep neural networks to be configurable and executed dynamically at runtime based on the resource constraints.…”

Section: Background and Related Workmentioning

confidence: 99%

Fusion-Catalyzed Pruning for Optimizing Deep Learning on Intelligent Edge Devices

Wang

et al. 2020

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

The increasing computational cost of deep neural network models limits the applicability of intelligent applications on resource-constrained edge devices. While a number of neural network pruning methods have been proposed to compress the models, prevailing approaches focus only on parametric operators (e.g., convolution), which may miss optimization opportunities. In this paper, we present a novel fusion-catalyzed pruning approach, called FUPRUNER, which simultaneously optimizes the parametric and non-parametric operators for accelerating neural networks. We introduce an aggressive fusion method to equivalently transform a model, which extends the optimization space of pruning and enables non-parametric operators to be pruned in a similar manner as parametric operators, and a dynamic filter pruning method is applied to decrease the computational cost of models while retaining the accuracy requirement. Moreover, FUPRUNER provides configurable optimization options for controlling fusion and pruning, allowing much more flexible performance-accuracy trade-offs to be made. Evaluation with state-of-the-art residual neural networks on five representative intelligent edge platforms, Jetson TX2, Jetson Nano, Edge TPU, NCS, and NCS2, demonstrates the effectiveness of our approach, which can accelerate the inference of models on CIFAR-10 and ImageNet datasets.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Fusion-Catalyzed Pruning for Optimizing Deep Learning on Intelligent Edge Devices

Wang

et al. 2020

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

show abstract

“…For instance, BinaryNet [Courbariaux and Bengio, 2016] limits the parameters to have 1-bit representations. Recently, [Jayakodi et al, 2020] proposed to design classifiers of increasing complexity using pre-trained Convolutional Neural Networks to perform input-specific adaptive inference.…”

Section: Model Compressionmentioning

confidence: 99%

“…Moreover, this very specific formulation is known as cardinality constrained knapsack or Exact K-item Knapsack Problem(E-KKP). This problem is proved to be NP-Complete [Kellerer et al, 2004].…”

Section: Theoretical Analysis Of the Problemmentioning

confidence: 99%

“…First, from a practical point of view, our problem size is not large and the exact solution can be computed in a reasonable time. Specifically, it is shown that the E-KKP problem with n items and U as the upper bound on the optimal solution value, can be solved to optimality by dynamic programming in O(nKU ) time and O(n+KU ) space [Kellerer et al, 2004]. Second, it is possible to apply linear programming relaxation techniques to achieve a near-optimal solution, which will be the topic of the next subsection.…”

Section: Theoretical Analysis Of the Problemmentioning

confidence: 99%

“…To address the aforementioned challenges, much effort has been made. Several special-purpose processor chips are designed to perform training or inference on edge devices with improved performance [Jouppi et al, 2017;Franklin, 2019]. Furthermore, techniques such as weight pruning [Zhu andGupta, 2017], quantization [Jacob et al, 2017], and knowledge distillation [Hinton et al, 2015] are developed to compress the models.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimal Policy for Deployment of Machine Learning Models on Energy-Bounded Systems

Mirzadeh

Ghasemzadeh

2020

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

With the recent advances in both machine learning and embedded systems research, the demand to deploy computational models for real-time execution on edge devices has increased substantially. Without deploying computational models on edge devices, the frequent transmission of sensor data to the cloud results in rapid battery draining due to the energy consumption of wireless data transmission. This rapid power dissipation leads to a considerable reduction in the battery lifetime of the system, therefore jeopardizing the real-world utility of smart devices. It is well-established that for difficult machine learning tasks, models with higher performance often require more computation power and thus are not power-efficient choices for deployment on edge devices. However, the trade-offs between performance and power consumption are not well studied. While numerous methods (e.g., model compression) have been developed to obtain an optimal model, these methods focus on improving the efficiency of a "single" model. In an entirely new direction, we introduce an effective method to find a combination of "multiple" models that are optimal in terms of power-efficiency and performance by solving an optimization problem in which both performance and power consumption are taken into account. Experimental results demonstrate that on the ImageNet dataset, we can achieve a 20% energy reduction with only 0.3% accuracy drop compared to Squeeze-and-Excitation Networks. Compared to a pruned convolutional neural network for human activity recognition, while consuming 1.7% less energy, our proposed policy achieves 1.3% higher accuracy.

show abstract

A systematic review of Green AI

Verdecchia

Sallou

Cruz

2023

WIREs Data Min & Knowl

View full text Add to dashboard Cite

With the ever‐growing adoption of artificial intelligence (AI)‐based systems, the carbon footprint of AI is no longer negligible. AI researchers and practitioners are therefore urged to hold themselves accountable for the carbon emissions of the AI models they design and use. This led in recent years to the appearance of researches tackling AI environmental sustainability, a field referred to as Green AI. Despite the rapid growth of interest in the topic, a comprehensive overview of Green AI research is to date still missing. To address this gap, in this article, we present a systematic review of the Green AI literature. From the analysis of 98 primary studies, different patterns emerge. The topic experienced a considerable growth from 2020 onward. Most studies consider monitoring AI model footprint, tuning hyperparameters to improve model sustainability, or benchmarking models. A mix of position papers, observational studies, and solution papers are present. Most papers focus on the training phase, are algorithm‐agnostic or study neural networks, and use image data. Laboratory experiments are the most common research strategy. Reported Green AI energy savings go up to 115%, with savings over 50% being rather common. Industrial parties are involved in Green AI studies, albeit most target academic readers. Green AI tool provisioning is scarce. As a conclusion, the Green AI research field results to have reached a considerable level of maturity. Therefore, from this review emerges that the time is suitable to adopt other Green AI research strategies, and port the numerous promising academic results to industrial practice. This article is categorized under: Technologies > Machine Learning

show abstract

Design and Optimization of Energy-Accuracy Tradeoff Networks for Mobile Platforms via Pretrained Deep Models

Cited by 23 publications

References 29 publications

Fusion-Catalyzed Pruning for Optimizing Deep Learning on Intelligent Edge Devices

Fusion-Catalyzed Pruning for Optimizing Deep Learning on Intelligent Edge Devices

Optimal Policy for Deployment of Machine Learning Models on Energy-Bounded Systems

A systematic review of Green AI

Contact Info

Product

Resources

About