Enhancing HEVC Compressed Videos with a Partition-Masked Convolutional Neural Network

He, Xin; Hu, Qiang; Zhang, Xiaoyun; Zhang, Chongyang; Lin, Weiyao; Han, Xintong

doi:10.1109/icip.2018.8451086

Cited by 81 publications

(53 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There have been several important contributions demonstrating the effectiveness of replacing or enhancing different components of traditional codecs with counterparts based on neural networks. These include improved motion compensation and interpolation [49,16,53,26], intra-prediction coding [35], post-processing refinement [7,50,21,51,43,52,36,14], and rate control [23].…”

Section: Ml-based Video Compressionmentioning

confidence: 99%

Learned Video Compression

Rippel

Nair

Lew

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

199

117

View full text Add to dashboard Cite

We present a new algorithm for video coding, learned end-to-end for the low-latency mode. In this setting, our approach outperforms all existing video codecs across nearly the entire bitrate range. To our knowledge, this is the first ML-based method to do so.We evaluate our approach on standard video compression test sets of varying resolutions, and benchmark against all mainstream commercial codecs, in the low-latency mode. On standard-definition videos, relative to our algorithm, HEVC/H.265, AVC/H.264 and VP9 typically produce codes up to 60% larger. On high-definition 1080p videos, H.265 and VP9 typically produce codes up to 20% larger, and H.264 up to 35% larger. Furthermore, our approach does not suffer from blocking artifacts and pixelation, and thus produces videos that are more visually pleasing.We propose two main contributions. The first is a novel architecture for video compression, which (1) generalizes motion estimation to perform any learned compensation beyond simple translations, (2) rather than strictly relying on previously transmitted reference frames, maintains a state of arbitrary information learned by the model, and (3) enables jointly compressing all transmitted signals (such as optical flow and residual).Secondly, we present a framework for ML-based spatial rate control -a mechanism for assigning variable bitrates across space for each frame. This is a critical component for video coding, which to our knowledge had not been developed within a machine learning setting.

show abstract

Section: Ml-based Video Compressionmentioning

confidence: 99%

Learned Video Compression

Rippel

Nair

Lew

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

199

117

View full text Add to dashboard Cite

show abstract

“…Previous in-loop filters designated for intracoded frames can be reused for single-frame postfiltering [162], [176]- [184]. Appropriate retraining may be applied in order to better capture the data characteristics.…”

Section: B Postfilteringmentioning

confidence: 99%

Advances in Video Compression System Using Deep Neural Network: A Review and Case Studies

Ding

Chen

et al. 2021

Proc. IEEE

View full text Add to dashboard Cite

Significant advances in video compression systems have been made in the past several decades to satisfy the near-exponential growth of Internet-scale video traffic. From the application perspective, we have identified three major functional blocks, including preprocessing, coding, and postprocessing, which have been continuously investigated to maximize the end-user quality of experience (QoE) under a limited bit rate budget. Recently, artificial intelligence (AI)-powered techniques have shown great potential to further increase the efficiency of the aforementioned functional blocks, both individually and jointly. In this article, we review recent technical advances in video compression systems extensively, with an emphasis on deep neural network (DNN)based approaches, and then present three comprehensive case studies. On preprocessing, we show a switchable texturebased video coding example that leverages DNN-based scene understanding to extract semantic areas for the improvement Manuscript

show abstract

“…In this case, the network architecture is complete, as shown in Fig. 2, but it is trained with λ G = 0 in (8). The results are shown as the red curve in Fig.…”

Section: A Pre-training With Ablation Studymentioning

confidence: 99%

Deep Frame Prediction for Video Coding

Choi

Bajić

2020

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

We propose a novel frame prediction method using a deep neural network (DNN), with the goal of improving video coding efficiency. The proposed DNN makes use of decoded frames, at both encoder and decoder, to predict textures of the current coding block. Unlike conventional inter-prediction, the proposed method does not require any motion information to be transferred between the encoder and the decoder. Still, both uni-directional and bi-directional prediction are possible using the proposed DNN, which is enabled by the use of the temporal index channel, in addition to color channels. In this study, we developed a jointly trained DNN for both uniand bi-directional prediction, as well as separate networks for uni-and bi-directional prediction, and compared the efficacy of both approaches. The proposed DNNs were compared with the conventional motion-compensated prediction in the latest video coding standard, HEVC, in terms of BD-Bitrate. The experiments show that the proposed joint DNN (for both uniand bi-directional prediction) reduces the luminance bitrate by about 4.4%, 2.4%, and 2.3% in the Low delay P, Low delay, and Random access configurations, respectively. In addition, using the separately trained DNNs brings further bit savings of about 0.3%-0.5%.

show abstract

Enhancing HEVC Compressed Videos with a Partition-Masked Convolutional Neural Network

Cited by 81 publications

References 12 publications

Learned Video Compression

Learned Video Compression

Advances in Video Compression System Using Deep Neural Network: A Review and Case Studies

Deep Frame Prediction for Video Coding

Contact Info

Product

Resources

About