Saliency-Driven Versatile Video Coding for Neural Object Detection

Fischer, Kristian; Fleckenstein, Felix; Herglotz, Christian; Kaup, André

doi:10.1109/icassp39728.2021.9415048

Cited by 25 publications

(14 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Saliency-driven image and video coding for humans has gained importance in the recent past. In [61] authors propose a saliency-driven coding framework for the video coding for machines task using the latest video coding standard Versatile Video Coding (VVC). To determine the salient regions before encoding, we employ the real-time-capable object detector YOLO in combination with a novel decision criterion.…”

Section: Video Coding Scheme For Specific Computer Vision Tasksmentioning

confidence: 99%

Task Oriented Video Coding: A Survey

Wood¹

2022

Preprint

View full text Add to dashboard Cite

Video coding technology has been continuously improved for higher compression ratio with higher resolution. However, the state-of-the-art video coding standards, such as H.265/HEVC and Versatile Video Coding, are still designed with the assumption the compressed video will be watched by humans. With the tremendous advance and maturation of deep neural networks in solving computer vision tasks, more and more videos are directly analyzed by deep neural networks without humans' involvement. Such a conventional design for video coding standard is not optimal when the compressed video is used by computer vision applications. While the human visual system is consistently sensitive to the content with high contrast, the impact of pixels on computer vision algorithms is driven by specific computer vision tasks. In this paper, we explore and summarize recent progress on computer vision task oriented video coding and emerging video coding standard, Video Coding for Machines.

show abstract

Section: Video Coding Scheme For Specific Computer Vision Tasksmentioning

confidence: 99%

Task Oriented Video Coding: A Survey

Wood¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Regarding (V) the number of considered distortions, existing studies consider at most 50 coding configurations, at the exception of Dejean-servieres et al [10]. This is because some studies only compare themselves to HEVC test Model (HM) or VVC Test Model (VTM) with few Quantization Parameter (QP) after proposing a new method to reach better tradeoffs between vision task performance and bitrate [14]- [17], [19], [25], [26]. Some papers also evaluate DNN resilience to JPEG/JPEG2000 compression [10], [12], [15], [18], [21], [24], [27], [28], [30], AVC [1], [31] or auto-encoders [18], [26], [27], but no paper consider all mentioned image and video codec generations in a unified framework (II).…”

Section: Related Workmentioning

confidence: 99%

Video Coding for Machines: Large-Scale Evaluation of Deep Neural Networks Robustness to Compression Artifacts for Semantic Segmentation

Marie

Desnos

Morin

et al. 2022

2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)

View full text Add to dashboard Cite

In the Video Coding for Machines (VCM) context where visual content is compressed before being transmitted to a vision task algorithm, appropriate trade-off between the compression level and the vision task performance must be chosen. In this paper, a Deep Neural Networks (DNN) based semantic segmentation algorithm robustness to compression artifacts is evaluated with a total of 1486 different coding configurations. Results indicate the importance of using an appropriate image resolution to overcome the block-partitioning limitations in existing compression algorithms, allowing 58.3%, 49.8%, 33.5% and 24.3% bitrate savings at equivalent prediction accuracy for JPEG, JM, x265 and VVenC, respectively. Surprisingly, JPEG can achieve 73.41% bitrate reduction with the inclusion of compressed images at training time over VVC Test Model (VTM) with a DNN trained on pristine data, which implies that DNN generalization ability must not be overlooked.

show abstract

“…Early work in this area optimized the compression to preserve extracted features [2,3,4]. More recent work focuses on coding for M2M communication, where a neural network analyzes the decoded image instead of a human observer [5,6,7,8,9,10]. This topic is commonly referred to as video coding for machines (VCM).…”

Section: Introductionmentioning

confidence: 99%

“…To the best of our knowledge, there is currently no dataset available that fulfills all three conditions. Thus, previous VCM research either sacrifices the condition of uncompressed input data [5,6] or evaluates the coding frameworks on single images [7,8,9,10].…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Video Coding for Machines without Ground Truth

Fischer

Hofbauer

Kuhn

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

In the emerging field of video coding for machines, video datasets with pristine video quality and high-quality annotations are required for a comprehensive evaluation. However, existing video datasets with detailed annotations are severely limited in size and video quality. Thus, current methods have to either evaluate their codecs on still images or on already compressed data. To mitigate this problem, we propose an evaluation method based on pseudo ground-truth data from the field of semantic segmentation to the evaluation of video coding for machines. Through extensive evaluation, this paper shows that the proposed ground-truth-agnostic evaluation method results in an acceptable absolute measurement error below 0.7 percentage points on the Bjøntegaard Delta Rate compared to using the true ground truth for mid-range bitrates. We evaluate on the three tasks of semantic segmentation, instance segmentation, and object detection. Lastly, we utilize the ground-truth-agnostic method to measure the coding performances of the VVC compared against HEVC on the Cityscapes sequences. This reveals that the coding position has a significant influence on the task performance.

show abstract

Saliency-Driven Versatile Video Coding for Neural Object Detection

Cited by 25 publications

References 16 publications

Task Oriented Video Coding: A Survey

Task Oriented Video Coding: A Survey

Video Coding for Machines: Large-Scale Evaluation of Deep Neural Networks Robustness to Compression Artifacts for Semantic Segmentation

Evaluation of Video Coding for Machines without Ground Truth

Contact Info

Product

Resources

About