Partitioning DNNs for Optimizing Distributed Inference Performance on Cooperative Edge Devices: A Genetic Algorithm Approach

Abstract

To fully unleash the potential of edge devices, it is popular to cut a neural network into multiple pieces and distribute them among available edge devices to perform inference cooperatively. Up to now, the problem of partitioning a deep neural network (DNN), which can result in the optimal distributed inferencing performance, has not been adequately addressed. This paper proposes a novel layer-based DNN partitioning approach to obtain an optimal distributed deployment solution. In order to ensure the applicability of the resulted deployment scheme, this work defines the partitioning problem as a constrained optimization problem and puts forward an improved genetic algorithm (GA). Compared with the basic GA, the proposed algorithm can result in a running time approximately one to three times shorter than the basic GA while achieving a better deployment.

Genetic Algorithm-Based Online-Partitioning BranchyNet for Accelerating Edge Inference

“…In turn, the camera will run the inference task assigned to it and send the corresponding output to the smartphone, which performs the second exit branch and obtains corresponding identification results. There are more similar application scenarios, such as distributed fall detection [ 8 ] and traffic prediction [ 9 ].…”

Optimizing DNNs Model Partitioning for Enhanced Performance on Edge Devices

“…Deploying DNN models on edge devices (e.g., embedded systems) presents various challenges, including the limited computational power and memory of edge devices, which can often prevent the deployment of large DNN models entirely. For example, Convolution Neural Networks (CNNs), another type of DNN, can be large and computationally intensive, which makes it challenging to deploy the entire CNN model on a single core edge device [6]. Traditionally, machine learning-based CPS applications were run sequentially on a singlecore processor or device [7].…”

“…For example, Chuang Hu et al [12] proposed a min-cut-based algorithm to partition and offload DNNs in both edge and cloud environments. Similarly, cloud and edge-assisted approaches [6,13] divide the DNNs into two parts to offer local and remote computation. However, previous studies have not addressed the ideal number of partitions for the DNN model concerning layer dependencies, device computing power, and communication latency.…”