Auto-Split: A General Framework of Collaborative Edge-Cloud AI

Banitalebi-Dehkordi, Amin; Vedula, Naveen; Pei, Jian; Xia, Fei; Wang, Lanjun; Zhang, Yong

doi:10.1145/3447548.3467078

Cited by 35 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the practical implementation, DADS relies on a modified version of Caffe. Similar to DADS, the industrial effort Auto-Split [21], and D 3 [22] are DAG-based. D 3 follows the three-layer (device, edge, cloud) concept of DDNN [12], as well as parallel distribution across edge nodes similar to, e.g., MoDNN [14].…”

Section: Related Workmentioning

confidence: 99%

Edge-PRUNE: Flexible Distributed Deep Learning Inference

Boutellier¹,

Tan²,

Nurmi³

2022

Preprint

View full text Add to dashboard Cite

Collaborative deep learning inference between lowresource endpoint devices and edge servers has received significant research interest in the last few years. Such computation partitioning can help reducing endpoint device energy consumption and improve latency, but equally importantly also contributes to privacy-preserving of sensitive data. This paper describes Edge-PRUNE, a flexible but light-weight computation framework for distributing machine learning inference between edge servers and one or more client devices. Compared to previous approaches, Edge-PRUNE is based on a formal dataflow computing model, and is agnostic towards machine learning training frameworks, offering at the same time wide support for leveraging deep learning accelerators such as embedded GPUs. The experimental section of the paper demonstrates the use and performance of Edge-PRUNE by image classification and object tracking applications on two heterogeneous endpoint devices and an edge server, over wireless and physical connections. Endpoint device inference time for SSD-Mobilenet based object tracking, for example, is accelerated 5.8× by collaborative inference.

show abstract

Section: Related Workmentioning

confidence: 99%

Edge-PRUNE: Flexible Distributed Deep Learning Inference

Boutellier¹,

Tan²,

Nurmi³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Perception latency and coverage can then be improved by offloading LIDAR data to edge servers for collaborative inference [10]. Although computational graph based model partitioning, together with dynamic scheduling, is general enough to be applied on any type of neural network, CNN workloads are often the intended use case [11], [12].…”

Section: B Edge Computingmentioning

confidence: 99%

Towards edge-assisted real-time 3D segmentation of large scale LIDAR point clouds

McLean

Xue

et al. 2022

Proceedings of the 6th International Workshop on Embedded and Mobile Deep Learning

View full text Add to dashboard Cite

“…4) Same features, different labels: The conditional distribution of labels given features may differ between edges. For example, the symbol EdgeRec System [28] Auto-Split [218] CoEdge [219] Colla [220] DCCL [30] MC 2 -SF [221] FedAvg [27] FML [222] Personalized FedAvg [223] HyperCluster [224] Federated Evaluation [225] represents correct in many countries and incorrect in some others (e.g., Japan); and 5) Quantity skew: Clients can store drastically varying volumes of data.…”

Section: Statistical Heterogeneitymentioning

confidence: 99%

“…Another successful practice is from the Taobao EdgeRec System [28], where the memoryconsuming embedding matrices encoding the attributes are deployed on the cloud side and the lightweight component executes the remaining inference on the edge side. Amin [218] introduced an Auto-Split solution to automatically split DNNs models into two parts respectively for the edge and for the cloud. Similarly, Hu [219] casts the split as a latency-minimum allocation problem and introduced an CoEdge solution.…”

Section: Split-deploymentmentioning

confidence: 99%

Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI

Yao¹,

Zhang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Influenced by the great success of deep learning via cloud computing and the rapid development of edge chips, research in artificial intelligence (AI) has shifted to both of the computing paradigms, i.e., cloud computing and edge computing. In recent years, we have witnessed significant progress in developing more advanced AI models on cloud servers that surpass traditional deep learning models owing to model innovations (e.g., Transformers, Pretrained families), explosion of training data and soaring computing capabilities. However, edge computing, especially edge and cloud collaborative computing, are still in its infancy to announce their success due to the resource-constrained IoT scenarios with very limited algorithms deployed. In this survey, we conduct a systematic review for both cloud and edge AI. Specifically, we are the first to set up the collaborative learning mechanism for cloud and edge modeling with a thorough review of the architectures that enable such mechanism. We also discuss potentials and practical experiences of some on-going advanced edge AI topics including pretraining models, graph neural networks and reinforcement learning. Finally, we discuss the promising directions and challenges in this field.

show abstract

Auto-Split: A General Framework of Collaborative Edge-Cloud AI

Cited by 35 publications

References 14 publications

Edge-PRUNE: Flexible Distributed Deep Learning Inference

Edge-PRUNE: Flexible Distributed Deep Learning Inference

Towards edge-assisted real-time 3D segmentation of large scale LIDAR point clouds

Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI

Contact Info

Product

Resources

About