Enabling microservices management for Deep Learning applications across the Edge-Cloud Continuum

Abstract

Deep Learning has shifted the focus of traditional batch workflows to data-driven feature engineering on streaming data. In particular, the execution of Deep Learning workflows presents expectations of near-real-time results with user-defined acceptable accuracy. Meeting the objectives of such applications across heterogeneous resources located at the edge of the network, the core, and in-between requires managing trade-offs between the accuracy and the urgency of the results. However, current data analysis rarely manages the entire Deep Learning pipeline along the data path, making it complex for developers to implement strategies in realworld deployments. Driven by an object detection use case, this paper presents an architecture for time-critical Deep Learning workflows by providing a data-driven scheduling approach to distribute the pipeline across Edge to Cloud resources. Furthermore, it adopts a data management strategy that reduces the resolution of incoming data when potential trade-off optimizations are available. We illustrate the system's viability through a performance evaluation of the object detection use case on the Grid'5000 testbed. We demonstrate that in a multi-user scenario, with a standard frame rate of 25 frames per second, the system speed-up data analysis up to 54.4% compared to a Cloud-only-based scenario with an analysis accuracy higher than a fixed threshold.

Task Allocation Methods and Optimization Techniques in Edge Computing: A Systematic Review of the Literature

“…Machine learning-based task allocation [28] DQN-D EC [54] IoT [38] MARL IoT [42] ACO IoT, 5G [51] FLOM-Opt [53] Q-Learning IoT, IoV [57] MA IoT [58] Heuristic, RL EC Quality of service task allocation [26] EC [32] MMAS EC [48] QT EC, IoT Resource-aware task allocation [76] MAPPO MEC [62] MDP EC [14] MAP VEC [75] JTORA VEC [72] JTORA MEC [74] JTORA MEC [15] DNF [16] MARL MEC [21] JTORA NOMA-MEC [37] EC [39] ELB [40] Knapsack MCS [70] EC, 5G…”

“…A work published by Zeina Houmani et al [54] addressed the issues with managing deep learning applications over an edge-cloud architecture where data are evaluated and transported between resources at the network's edge. Trade-offs between accuracy and latency are important for DL applications since they need outcomes close to real-time with a precision that the user has set as acceptable.…”

MDED-Framework: A Distributed Microservice Deep-Learning Framework for Object Detection in Edge Computing

“…Apache Kafka [ 19 ] was used to introduce a system that can offer feedback on real-time queries. Houmani Z. et al [ 20 ] proposed a microservice resource-management scheduling method for deep learning applications that overview edge cloud. The study proposes a deep learning workflow architecture that divides cloud resources into three categories (non-intensive, low-intensive, and high-intensive) based on CPU, memory, storage, or bandwidth requirements, and which uses distributed pipelining.…”