Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

Abstract

As a key technology of enabling Artificial Intelligence (AI) applications in 5G era, Deep Neural Networks (DNNs) have quickly attracted widespread attention. However, it is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources. What's worse, traditional cloud-assisted DNN inference is heavily hindered by the significant wide-area network latency, leading to poor real-time performance as well as low quality of user experience. To address these challenges, in this paper, we propose Edgent, a framework that leverages edge computing for DNN collaborative inference through device-edge synergy. Edgent exploits two design knobs: (1) DNN partitioning that adaptively partitions computation between device and edge for purpose of coordinating the powerful cloud resource and the proximal edge resource for real-time DNN inference; (2) DNN right-sizing that further reduces computing latency via early exiting inference at an appropriate intermediate DNN layer. In addition, considering the potential network fluctuation in realworld deployment, Edgent is properly design to specialize for both static and dynamic network environment. Specifically, in a static environment where the bandwidth changes slowly, Edgent derives the best configurations with the assist of regressionbased prediction models, while in a dynamic environment where the bandwidth varies dramatically, Edgent generates the best execution plan through the online change point detection algorithm that maps the current bandwidth state to the optimal configuration. We implement Edgent prototype based on the Raspberry Pi and the desktop PC and the extensive experimental evaluations demonstrate Edgent's effectiveness in enabling ondemand low-latency edge intelligence.

Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework

“…Specifically, the collaborative learning process in PerFit mainly consists of the following three stages as depicted in r Offloading stage: When the edge is trustworthy (e.g., edge gateway at home), the IoT device user can offload its whole learning model and data samples to the edge for fast computation. Otherwise, the device user will carry out model partitioning by keeping the input layers and its data samples locally on its device and offloading the remaining model layers to the edge for device-edge collaborative computing [14].…”

Blockchain-Based Incentive Energy-Knowledge Trading in IoT: Joint Power Transfer and AI Design

“…Once appeared, it received a tremendous amount of interest. In terms of edge AI algorithms, in [22] [23], Li et al proposed an Edgent framework, which leverages deviceedge synergy computing to accelerate training and inference of deep neural network (DNN). And the authors in [24] design a momentum federated learning (MFL) framework to accelerate federated learning by momentum gradient descent.…”

Crowd-MECS: A Novel Crowdsourcing Framework for Mobile Edge Caching and Sharing

“…We call such a scenario as the Non-MEC system, and will compare the two systems in the simulation section. 6 2) CP's Profit Maximization in Stage I: To maximize the CP's profit regarding the revenue sharing ratio, we need to first analyze the total profit of the CP. Recall that p is the price of per content request paid to the CP, s CD is the CP's delivery cost of per content request, and I · (|I RE | − B|I AG |) is the extra requesters' demands that exceed the agents' total serving capacities.…”