X-Leep

Herzog, Benedict; Pereira, Pablo Gil; Schmidt, Andreas; Büttner, Tobias; Hönig, Timo; Schröder-Preikschat, Wolfgang; Herfet, Thorsten

doi:10.1145/3396851.3402924

Cited by 2 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep models such as MaskTrack R-CNN [2] are widely employed for multi-task video analytics, such as object detection, object classification, and segmentation. Deep models are generally energy-intensive due to the high amount of video stream data to process, which constrains their adoption in energy-constrained scenarios such as edge computing [4]. However, the ability to perform intelligent video analytics in energy-constrained edge devices is becoming increasingly important with the fast expansion of intelligent Internet-of-Things [4], [5].…”

Section: Introductionmentioning

confidence: 99%

“…Deep models are generally energy-intensive due to the high amount of video stream data to process, which constrains their adoption in energy-constrained scenarios such as edge computing [4]. However, the ability to perform intelligent video analytics in energy-constrained edge devices is becoming increasingly important with the fast expansion of intelligent Internet-of-Things [4], [5]. There is an urgent need for energy-efficient multi-task video analytics.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Reinforcement-Learning-Based Energy-Efficient Framework for Multi-Task Video Analytics Pipeline

Zhao

Dong

Wang

et al. 2022

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Deep-learning-based video processing has yielded transformative results in recent years. However, the video analytics pipeline is energy-intensive due to high data rates and reliance on complex inference algorithms, which limits its adoption in energy-constrained applications. Motivated by the observation of high and variable spatial redundancy and temporal dynamics in video data streams, we design and evaluate an adaptive-resolution optimization framework to minimize the energy use of multitask video analytics pipelines. Instead of heuristically tuning the input data resolution of individual tasks, our framework utilizes deep reinforcement learning to dynamically govern the input resolution and computation of the entire video analytics pipeline. By monitoring the impact of varying resolution on the quality of high-dimensional video analytics features, hence the accuracy of video analytics results, the proposed end-to-end optimization framework learns the best non-myopic policy for dynamically controlling the resolution of input video streams to globally optimize energy efficiency. Governed by reinforcement learning, optical flow is incorporated into the framework to minimize unnecessary spatio-temporal redundancy that leads to re-computation, while preserving accuracy. The proposed framework is applied to video instance segmentation which is one of the most challenging computer vision tasks, and achieves better energy efficiency than all baseline methods of similar accuracy on the YouTube-VIS dataset.

show abstract