A Serverless Cloud-Fog Platform for DNN-Based Video Analytics with Incremental Learning

Abstract

Deep neural networks (DNN) based video analytics have empowered many new applications, such as automated retail and smart city. Meanwhile, the proliferation of fog computing systems provides system developers with more design options to improve performance and save cost. To the best of our knowledge, this paper presents the first serverless system that takes full advantage of the client-fog-cloud synergy to better serve the DNN-based video analytics. Specifically, the system aims to achieve two goals: 1) Provide the optimal analytics results under the constraints of lower bandwidth usage and shorter roundtrip time (RTT) by judiciously managing the computational and bandwidth resources deployed in the client, fog, and cloud environment. 2) Free developers from tedious administration and operation tasks, including DNN deployment, cloud and fog's resource management. To this end, we design and implement a holistic cloud-fog system referred to as VPaaS (Video-Platformas-a-Service) to execute inference related tasks. The proposed system adopts serverless computing to enable developers to build a video analytics pipeline by simply programming a set of functions (e.g., encoding and decoding, and model inference). These functions are then orchestrated to process video streaming through carefully designed modules. To save bandwidth and reduce RTT at the same time, VPaaS provides a new video streaming protocol that only sends low-quality video to the cloud. The state-of-the-art hardware accelerators and high-performing DNNs deployed at the cloud can identify regions of video frames that need further processing at the fog ends. At the fog ends, misidentified labels in these regions can be corrected using a light-weight DNN model. To address the data drift issues, we incorporate limited human feedback into the system to verify the results and adopt incremental machine learning to improve our system continuously. The evaluation of our system with extensive experiments on standard video datasets demonstrates that VPaaS is superior to several state-of-the-art systems: it maintains high accuracy while reducing bandwidth usage by up to 21%, RTT by up to 62.5%, and cloud monetary cost by up to 50%. We plan to release VPaaS as open-source software to facilitate the research and development of video analytics.

Efficiency in the serverless cloud paradigm: A survey on the reusing and approximation aspects

“…It is noteworthy that incremental reusing is a highly context-specific technique, and currently, it is not implemented within the general-purpose serverless platforms. For instance, Zhang et al 117 propose a serverless and FaaS-based platform that takes advantage of incremental computing in the video analytics context. Their use case employs a deep neural network (DNN) model for video object classification.…”

Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges

“…To narrow the domain shifts for the training and deployment stages, some recent works [4]- [7] adopted the lifelong learning strategies for person ReID. Lifelong learning enables the models to continuously learn from every new domain or scenario, which has been widely adopted in many DNN-based serving systems [25]- [27] to deal with the domain drift. The greatest challenge for lifelong person ReID is catastrophic forgetting, which requires ReID models to retain previous knowledge while continually training on new task streams.…”

Efficiency in the Serverless Cloud Computing Paradigm: A Survey Study

“…It is noteworthy that incremental reusing is a highly context-specific technique and currently it is not implemented within the general-purpose serverless platforms. For instance, Zhang et al, [87] propose a serverless and FaaSbased platform that takes advantage of incremental computing the video analytics context. Their use case employs a deep neural network (DNN) model for video object classification.…”