Visual saliency based perceptual video coding in HEVC

Wei, Henglu; Zhou, Xin; Zhou, Wei; Yan, Chao; Duan, Zhemin; Shan, Nana

doi:10.1109/iscas.2016.7539112

Cited by 14 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the method of [63], the BD-rate and BD-PSNR are 4.6 and 5.4, respectively, which are much higher than those of baseline HM16.19 coding algorithms, including [59] and [115]. When a recently proposed JND-based perceptual RC method for HEVC [66] is compared with [116] and [117], it can be seen that this method significantly reduces the bit rate while ensuring the subjective video quality. In addition, in terms of code control, the actual R of the encoding is closer to the target R.…”

Section: B Implementations At the Block Levelmentioning

confidence: 90%

Recent Advances in Rate Control: From Optimization to Implementation and Beyond

Wei¹,

Zhou²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Video coding is a video compression technique that compresses the original video sequence to achieve a smaller archive file or a lower transmission bandwidth under constraints on the visual quality loss. Rate control (RC) plays a critical role in video coding. It can achieve stable stream output in practical applications, especially in real-time video applications such as video conferencing or game live streaming. Most RC algorithms directly or indirectly characterize the relationship between the bit rate (R) and quantization (Q) and then allocate bits for every coding unit to guarantee the global bit rate and video quality level. This paper comprehensively reviews the classic RC technologies used in international video standards of past generations, analyses the mathematical models and implementation mechanisms of various schemes, and compares the performance of recent state-of-the-art RC algorithms. Finally, we discuss future directions and new application areas for RC methods. We hope that this review can help support the development, implementation, and application of RC in new video coding standards.

show abstract

Section: B Implementations At the Block Levelmentioning

confidence: 90%

Recent Advances in Rate Control: From Optimization to Implementation and Beyond

Wei¹,

Zhou²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent years, various video coding techniques have been proposed that used different techniques to code certain regions in high-quality and certain regions in low-quality [42][43][44][45][46][47][48][49][50][51]. Xu et al [42] proposed HEVC-based rate control scheme for video conferencing systems to encode facial features in high quality and background regions in lower quality.…”

Section: Background and Related Workmentioning

confidence: 99%

“…The different ROI-based video coding systems discussed above [47][48][49] demonstrated good results; however, manually selecting the ROI is a hindrance and causes inconvenience. Instead of manually selecting ROI, Wei et al [50] proposed a visual saliency-based coding scheme using HEVC. Visual saliency map was used for allocating coding bits or controlling video quality based on the probability of human attention over frames.…”

Section: Background and Related Workmentioning

confidence: 99%

High Efficiency Video Coding (HEVC)–Based Surgical Telementoring System Using Shallow Convolutional Neural Network

et al. 2019

View full text Add to dashboard Cite

Surgical telementoring systems have gained lots of interest, especially in remote locations. However, bandwidth constraint has been the primary bottleneck for efficient telementoring systems. This study aims to establish an efficient surgical telementoring system, where the qualified surgeon (mentor) provides real-time guidance and technical assistance for surgical procedures to the on-spot physician (surgeon). High Efficiency Video Coding (HEVC/H.265)-based video compression has shown promising results for telementoring applications. However, there is a trade-off between the bandwidth resources required for video transmission and quality of video received by the remote surgeon. In order to efficiently compress and transmit real-time surgical videos, a hybrid lossless-lossy approach is proposed where surgical incision region is coded in high quality whereas the background region is coded in low quality based on distance from the surgical incision region. For surgical incision region extraction, state-of-the-art deep learning (DL) architectures for semantic segmentation can be used. However, the computational complexity of these architectures is high resulting in large training and inference times. For telementoring systems, encoding time is crucial; therefore, very deep architectures are not suitable for surgical incision extraction. In this study, we propose a shallow convolutional neural network (S-CNN)-based segmentation approach that consists of encoder network only for surgical region extraction. The segmentation performance of S-CNN is compared with one of the state-of-the-art image segmentation networks (SegNet), and results demonstrate the effectiveness of the proposed network. The proposed telementoring system is efficient and explicitly considers the physiological nature of the human visual system to encode the video by providing good overall visual impact in the location of surgery. The results of the proposed S-CNN-based segmentation demonstrated a pixel accuracy of 97% and a mean intersection over union accuracy of 79%. Similarly, HEVC experimental results showed that the proposed surgical region-based encoding scheme achieved an average bitrate reduction of 88.8% at high-quality settings in comparison with default full-frame HEVC encoding. The average gain in encoding performance (signal-to-noise) of the proposed algorithm is 11.5 dB in the surgical region. The bitrate saving and visual quality of the proposed optimal bit allocation scheme are compared with the mean shift segmentation-based coding scheme for fair comparison. The results show that the proposed scheme maintains high visual quality in surgical incision region along with achieving good bitrate saving. Based on comparison and results, the proposed encoding algorithm can be considered as an efficient and effective solution for surgical telementoring systems for low-bandwidth networks.

show abstract

“…In the recent past, saliency-driven coding frameworks for a human observer have already been investigated for Joint Photographic Experts Group (JPEG) and High-Efficiency Video Coding (HEVC) in [7] and [8], respectively. Using saliency detection algorithms to improve the coding performance for the VCM task has also been researched for older coding standards than VVC.…”

Section: Introductionmentioning

confidence: 99%

Saliency-Driven Versatile Video Coding for Neural Object Detection

Fischer

Fleckenstein

Herglotz

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Saliency-driven image and video coding for humans has gained importance in the recent past. In this paper, we propose such a saliency-driven coding framework for the video coding for machines task using the latest video coding standard Versatile Video Coding (VVC). To determine the salient regions before encoding, we employ the real-time-capable object detection network You Only Look Once (YOLO) in combination with a novel decision criterion. To measure the coding quality for a machine, the state-of-the-art object segmentation network Mask R-CNN was applied to the decoded frame. From extensive simulations we find that, compared to the reference VVC with a constant quality, up to 29 % of bitrate can be saved with the same detection accuracy at the decoder side by applying the proposed saliency-driven framework. Besides, we compare YOLO against other, more traditional saliency detection methods.

show abstract

Visual saliency based perceptual video coding in HEVC

Cited by 14 publications

References 9 publications

Recent Advances in Rate Control: From Optimization to Implementation and Beyond

Recent Advances in Rate Control: From Optimization to Implementation and Beyond

High Efficiency Video Coding (HEVC)–Based Surgical Telementoring System Using Shallow Convolutional Neural Network

Saliency-Driven Versatile Video Coding for Neural Object Detection

Contact Info

Product

Resources

About