Energy-Efficient Service Placement for Latency-Sensitive Applications in Edge Computing

Abstract

Edge computing is a promising solution to host artificial intelligence (AI) applications that enable real-time insights on user-generated and device-generated data. This requires edge computing resources (storage and compute) to be widely deployed close to end devices. Such edge deployments require a large amount of energy to run as edge resources are typically overprovisioned to flexibly meet the needs of time-varying user demand with a low latency. Moreover, AI applications rely on deep neural network (DNN) models that are increasingly larger in size to support high accuracy. These DNN models must be efficiently stored and transferred, so as to minimize their energy consumption. In this article, we model the problem of energy-efficient placement of services (namely, DNN models) for AI applications as a multiperiod optimization problem. The formulation jointly places services and schedules requests such that the overall energy consumption is minimized and latency is low. We propose a heuristic that efficiently solves the problem while taking into account the impact of placing services across time periods. We assess the quality of the proposed heuristic by comparing its solution to a lower bound of the problem, obtained by formulating and solving a Lagrangian relaxation of the original problem. Extensive simulations show that our proposed heuristic outperforms baseline approaches in achieving a low energy consumption by packing services on a minimal number of edge nodes, while at the same time keeping the average latency of served requests below a configured threshold in nearly all time periods.

Dynamic Service Provisioning in Heterogenous Fog Computing Architecture Using Deep Reinforcement Learning

“…𝐷(𝑠𝑙𝑎 𝑖.𝑎 ) = 𝑏 𝑎 * 𝑠𝑙𝑎 𝑖.𝑎 + 𝑣 𝑎 (32) The parameters 𝑏 𝑎 and 𝑣 𝑎 , which are negative and positive, respectively, are parametrically defined for each service [18].…”

“…In this field, only a limited number of studies have tackled the challenges of service placement or user traffic routing, specifically specially focusing on certain aspects of the heterogeneity of fog nodes [32][33][34][35][36][37][38]. However, none of the existing research has addressed the problem of user traffic routing and service placement in the fog computing layer, aiming to minimize the service provisioning cost, particularly in the presence of ad hoc and fixed (dedicated) fog computing nodes.…”

Bayesian Optimization for Efficient Heterogeneous MPSoC Based DNN Accelerator Runtime Tuning

“…In recent years, Deep Neural Networks (DNN) have become one of the most commonly used machine learning (ML) algorithms in a wide range of AI applications. The development towards deeper neural networks results in higher computational requirements, which make it difficult to implement large models on a resource-constrained edge device with a satisfying computing speed, memory, and energy cost requirements [1]. To overcome these challenges, pruning network structure and quantizing floating-point precision and various pruning techniques have become one of the most promising offline solutions for reducing the computation consumption of inference with acceptable accuracy loss [2].…”