VideoEdge: Processing Camera Streams using Hierarchical Clusters

Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing

“…The follow-up work Deep Compression [71] which blends the advantages of pruning, weight sharing and Huffman coding to compress DNNs, further pushes the compression ratio to 35-49x. However, for energy-constrained end devices, the above magnitude-based weight pruning method may not be directly applicable, since empirical measurements show that the reduction of the number of weights does not necessarily translate into significant energy saving [72]. This is because for DNNs as exemplified by AlexNet, the energy of the convolutional layers dominates the total energy cost, while the number in the fully-connected layers contributes most of the total number of Model Partition • Computation offloading to the edge server or mobile devices • Latency-and energy-oriented optimization [10], [78]- [86] Model Early-Exit • Partial DNNs model inference • Accuracy-aware [10], [15], [78], [87]- [91] Edge Caching • Fast response towards reusing the previous results of the same task [92]- [96] Input Filtering • Detecting difference between inputs, avoiding abundant computation [97]- [101] Model Selection • Inputs-oriented optimization • Accuracy-aware [102]- [106] Support for Multi-Tenancy • Scheduling multiple DNN-based task • Resource-efficient [38], [104], [107]- [111] Application-specific Optimization • Optimizations for the specific DNN-based application • Resource-efficient [104], [112] weights in the DNN. This suggests that the number of weights may not be a good indicator for energy, and the weight pruning should be directly energy-aware for end devices.…”

Multi-tenant mobile offloading systems for real-time computer vision applications

“…This dissertation targets mobile offloading techniques at the emerging mobile vision applications. We focus on servers deployed in a cloudlet that provide low delay [56,37,38] and reduce the bandwidth usage for streaming visual data to the cloud [162,72]. Before introducing our work, we first identify key open questions in supporting such applications.…”

“…As a remedy, easy-to-use APIs to build the application and an underlying distributed system to deploy processing modules as microservices can improve the efficiency of offloading more complex applications. Although vision analytics platforms [160,100,72] present APIs and systems of this type, these target at cloud-scaled video analytics workload for stationary cameras. Essential issues such as the support for sub-second level real-time (RT) workloads, integration with mobile computing platforms and executing DNN models on GPUs remain untapped currently.…”

“…As we have introduced in Section 3.2, processing video data in the cloud, which is a special class of big data workload, has been studied in previous works [100,160,72]. For example, Optasia [100] builds vision application using the SCOPE [35] dataflow engine, with design patterns including extractors, processors, reducers, and combiners.…”

“…The profiler and scheduler components on the master server is responsible for determining query configuration and placement. VideoEdge [72] further explores the query selection issues for a hierarchical cluster including cameras, private clusters, and public clouds.…”