Joint Multiuser DNN Partitioning and Computational Resource Allocation for Collaborative Edge Intelligence

Abstract

Mobile Edge Computing (MEC) has emerged as a promising supporting architecture providing a variety of resources to the network edge, thus acting as an enabler for edge intelligence services empowering massive mobile and Internet of Things (IoT) devices with AI capability. With the assistance of edge servers, user equipments (UEs) are able to run deep neural network (DNN) based AI applications, which are generally resource-hungry and compute-intensive, such that an individual UE can hardly afford by itself in real time. However the resources in each individual edge server are typically limited. Therefore, any resource optimization involving edge servers is by nature a resource-constrained optimization problem and needs to be tackled in such realistic context. Motivated by this observation, we investigate the optimization problem of DNN partitioning (an emerging DNN offloading scheme) in a realistic multiuser resource-constrained condition that rarely considered in previous works. Despite the extremely large solution space, we reveal several properties of this specific optimization problem of joint multi-UE DNN partitioning and computational resource allocation. We propose an algorithm called Iterative Alternating Optimization (IAO) that can achieve the optimal solution in polynomial time. In addition, we present rigorous theoretic analysis of our algorithm in terms of time complexity and performance under realistic estimation error. Moreover, we build a prototype that implements our framework and conduct extensive experiments using realistic DNN models, whose results demonstrate its effectiveness and efficiency.

Latency-Memory Optimized Splitting of Convolution Neural Networks for Resource Constrained Edge Devices

“…Another class of approaches splits the CNN architecture between the edge device and the server [19,22,23,29,32]. The algorithmbased splitting methods decide the splitting point either based on a model-specific threshold [23] or input-output dimension size [22].…”

“…However, these are model-specific splitting and cannot be generalized. Other approaches split the CNN by trying to optimize the computation latency [29,32] limiting the utility of the edge device.…”

“…These approaches are limited by the level of image splitting and would thus be model-specific. Instead of carrying out all the computations on the edge device, the other set of solutions split the CNN architecture to perform part of the processing on the cloud [23,29]. However, these works either split the CNN based on some model-specific empirical thresholds or rely on latency optimization, which in most cases would defer the splitting.…”

SmartSplit: Latency-Energy-Memory Optimisation for CNN Splitting on Smartphone Environment

“…Another popular approach is splitting the CNN between the smartphone and a cloud server where a part of the processing is done on the cloud server. Such splitting could be done using N-step algorithms [13] or using latency-based optimisation [14]. These approaches have three major shortcomings when applied to smartphones.…”

“…Smartphones are used for performing concurrent tasks that need to be taken into consideration. Second, these approaches optimise only a single system parameter, be it memory utilisation [13] or latency [14]. However, considering the multi-tasking capability of smartphones, we need to optimise both the parameters.…”