Offloading Algorithms for Maximizing Inference Accuracy on Edge Device Under a Time Constraint

Abstract

With the emergence of edge computing, the problem of offloading jobs between an Edge Device (ED) and an Edge Server (ES) received significant attention in the past. Motivated by the fact that an increasing number of applications are using Machine Learning (ML) inference from the data samples collected at the EDs, we study the problem of offloading inference jobs by considering the following novel aspects: 1) in contrast to a typical computational job, the processing time of an inference job depends on the size of the ML model, and 2) recently proposed Deep Neural Networks (DNNs) for resource-constrained devices provide the choice of scaling down the model size by trading off the inference accuracy. Considering that multiple ML models are available at the ED, and a powerful ML model is available at the ES, we formulate a general assignment problem with the objective of maximizing the total inference accuracy of n data samples at the ED subject to a time constraint T on the makespan. Noting that the problem is NP-hard, we propose an approximation algorithm Accuracy Maximization using LP-Relaxation and Rounding (AMR 2 ), and prove that it results in a makespan at most 2T , and achieves a total accuracy that is lower by a small constant from the optimal total accuracy. Further, if the data samples are identical we propose Accuracy Maximization using Dynamic Programming (AMDP), an optimal pseudo-polynomial time algorithm. As proof of concept, we implemented AMR 2 on a Raspberry Pi, equipped with MobileNets, that is connected to a server equipped with ResNet, and studied the total accuracy and makespan performance of AMR 2 for image classification.

Energy-Aware Selective Inference Task Offloading for Real-Time Edge Computing Applications

“…The authors in [8] proposed AM R 2 , a scheduling scheme based on LP-Relaxation and rounding which considers all possible cases of scheduling two inference tasks between the edge device and the edge server. They relax the problem's constraint to take fractional values and then perform rounding to get the result.…”

“…This flexibility allows edge nodes to deploy multiple local inference models, each optimized for different accuracy levels. [8] Considering an edge computing system empowered by AI models in which edge devices receive sensed data from the en-vironment or from user interactions (see Fig 1). These devices are equipped with pretrained inference models which can be used to perform inference on data locally.…”