A Fidelity-Assured Rate Distortion Optimization Method for Perceptual-Based Video Coding

Wang, Qun; Yuan, Hui; Huo, Junyan; Li, Peng

doi:10.1109/icip.2019.8803496

Cited by 4 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, approximated 𝐷 𝑆𝑆𝐼 𝑀 -𝐷 𝑀𝑆𝐸 models in [25,50,120] can also be applicable to the PRDO. In [86], input video was divided into two classes, i.e., simple/regular texture regions (more sensitive) and complex texture regions (less sensitive), based on the free-energy principle. Then, a weighted combination of fidelity (𝐷 𝑆𝑆𝐸 ) and perceptual distortion (1 − 𝐷 𝑆𝑆𝐼 𝑀 ), i.e., D V = 𝜔 (1 − 𝐷 𝑆𝑆𝐼 𝑀 ) + 𝐷 𝑀𝑆𝐸 , was used as the final distortion metric in the PRDO, where larger weight 𝜔 was given to the perceptual distortion for sensitive regions with simple/regular texture.…”

Section: Perceptual Rate Distortion Optimization (Prdo)mentioning

confidence: 99%

“…The general workflow of the perceptual RDO is adding the perceptual quality metric or visual properties, such as JND [2,39], masking effect [16] and saliency [68], to the distortion term of the RD cost function, then, Lagrangian multiplier is adjusted correspondingly based on the perceptual distortion analysis. For some methods, such as [68,86], the quantization parameter is adjusted accordingly to allocate the bit rate more reasonably. By using this perceptual RDO criteria, smaller partitioning, refined block modes or motion parameters were allocated to perceptually significant regions, and coarse modes or partitions will be used for the perceptually insensitive regions.…”

Section: Perceptual Rate Distortion Optimization (Prdo)mentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Perceptually Optimized Video Coding

Zhang¹,

Zhu²,

Jiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Videos are developing in the trends of Ultra High Definition (UHD), High Frame Rate (HFR), High Dynamic Range (HDR), Wide Color Gammut (WCG) and high fidelity, which provide users with more realistic visual experiences. However, the amount of video data increases exponentially and requires high efficiency video compression for storage and network transmission. Perceptually optimized video coding aims to exploit visual redundancies in videos so as to maximize compression efficiency. In this paper, we present a systematic survey on the recent advances and challenges on perceptually optimized video coding. Firstly, we present problem formulation and framework of perceptually optimized video coding, which includes visual perception modelling, visual quality assessment and perception guided coding optimization. Secondly, the recent advances on visual factors, key computational visual models and quality assessment models are presented. Thirdly, we do systematic review on perceptual video coding optimizations from four key aspects, which includes perceptually optimized bit allocation, rate-distortion optimization, transform and quantization, filtering and enhancement. In each part, problem formulation, working flow, recent advances, advantages and challenges are presented. Fourthly, perceptual coding performance of latest coding standards and tools are experimentally analyzed. Finally, challenging issues and future opportunities on perceptual video coding are identified.CCS Concepts: • Computing methodologies → Image compression.

show abstract

Section: Perceptual Rate Distortion Optimization (Prdo)mentioning

confidence: 99%

Section: Perceptual Rate Distortion Optimization (Prdo)mentioning

confidence: 99%

A Survey on Perceptually Optimized Video Coding

Zhang¹,

Zhu²,

Jiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Multiple approaches already exist aiming to optimize the RDO for different use case scenarios. In [13], the authors stated that the visual impression of the human is different from the image fidelity which is measured by the SSE. Therefore, they used a hybrid approach combining the SSE distortion measurement with the structural similarity index (SSIM) to increase the subjective quality while still preserving the image fidelity.…”

Section: Rate Distortion Optimization and Block Partitioning In Vvc A...mentioning

confidence: 99%

“…With this normalized distortion and since it was shown in [13] and [15] that it is beneficial to not completely omit the common SSE as distortion, we also propose a hybrid FRDO (HFRDO) version that equally combines D SSE and a normalized feature-based distortion D * F by…”

Section: Adapt Feature-based Distortion Measure To λmentioning

confidence: 99%

Video Coding for Machines with Feature-Based Rate-Distortion Optimization

Fischer

Brand

Herglotz

et al. 2020

2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP)

View full text Add to dashboard Cite

Common state-of-the-art video codecs are optimized to deliver a low bitrate by providing a certain quality for the final human observer, which is achieved by rate-distortion optimization (RDO). But, with the steady improvement of neural networks solving computer vision tasks, more and more multimedia data is not observed by humans anymore, but directly analyzed by neural networks. In this paper, we propose a standard-compliant feature-based RDO (FRDO) that is designed to increase the coding performance, when the decoded frame is analyzed by a neural network in a video coding for machine scenario. To that extent, we replace the pixel-based distortion metrics in conventional RDO of VTM-8.0 with distortion metrics calculated in the feature space created by the first layers of a neural network. Throughout several tests with the segmentation network Mask R-CNN and single images from the Cityscapes dataset, we compare the proposed FRDO and its hybrid version HFRDO with different distortion measures in the feature space against the conventional RDO. With HFRDO, up to 5.49 % bitrate can be saved compared to the VTM-8.0 implementation in terms of Bjøntegaard Delta Rate and using the weighted average precision as quality metric. Additionally, allowing the encoder to vary the quantization parameter results in coding gains for the proposed HFRDO of up 9.95 % compared to conventional VTM.

show abstract