iCUS: Intelligent CU Size Selection for HEVC Inter Prediction

Abstract

The hierarchical quadtree partitioning of Coding Tree Units (CTU) is one of the striking features in HEVC that contributes towards its superior coding performance over its predecessors. However, the brute force evaluation of the quadtree hierarchy using the Rate-Distortion (RD) optimisation, to determine the best partitioning structure for a given content, makes it one of the most time-consuming operations in HEVC encoding. In this context, this paper proposes an intelligent fast Coding Unit (CU) size selection algorithm to expedite the encoding process of HEVC inter-prediction. The proposed algorithm introduces (i) two CU split likelihood modelling and classification approaches using Support Vector Machines (SVM) and Bayesian probabilistic models, and (ii) a fast CU selection algorithm that makes use of both offline trained SVMs and online trained Bayesian probabilistic models. Finally, (iii) a computational complexity to coding efficiency trade-off mechanism is introduced to flexibly control the algorithm to suit different encoding requirements. The experimental results of the proposed algorithm demonstrate an average encoding time reduction performance of 53.46%, 61.15%, and 58.15% for Low Delay B, Random Access, and Low Delay P configurations, respectively, with Bjøntegaard Delta-Bit Rate (BD-BR) losses of 2.35%, 2.9%, and 2.35%, respectively, when evaluated across a wide range of content types and quality levels. INDEX TERMS Coding Unit (CU), encoder complexity reduction, High Efficiency Video Coding (HEVC), inter-prediction, Support Vector Machine (SVM) R ECENT advancements in multimedia technologies that span across Consumer Electronics (CE) in video content capturing, transmission and display have made video data the most frequently exchanged type of content over the modern communication networks. The increasing mobile consumption of High Definition (HD) and Ultra High Definition (UHD) video contents has contributed immensely towards the ever-growing IP video traffic and it is expected to reach over 82% of the overall Internet traffic in 2021 [1]. However, the estimated growth in network bandwidth (1.9 fold from 2017-2022, which is 39.0 Mbps to 75.4 Mbps for fixed broadband [1]) with time is still not sufficient to cater for the ever-growing user demands. Furthermore, the video requirements for emerging applications such as Augmented Reality (AR)/Virtual Reality (VR), interactive television, multi-party video conferences and over-the-top (OTT) multimedia consumption demand continuous improvements in the compression efficiency [2]. In this regard, High Efficiency Video Coding (HEVC) which was introduced in 2013 is the most recent stable video coding standard. It provides greater compression efficiency through an assortment of new features and coding tools over its predecessor H.264/AVC [3]. Out of these, the hierarchical quadtree partitioning structure introduced in HEVC that entails a wide range of Coding Unit (CU) sizes (i.e., 8 × 8 to 64 × 64) and their combinations, is one of the important contributors...

Measuring, Modeling and Integrating Time-Varying Video Quality in End-to-End Multimedia Service Delivery: A Review and Open Challenges

“…Therefore, state-of-the-art methods that focus on encoder complexity reduction attempt to skip all or certain stages in the compute intensive RD optimization when determining the best possible encoding parameter combinations for a given content. In this regard, the majority of the encoder complexity reduction methods can be categorized into two approaches; statistical feature-based methods and learningbased approaches [154]. Statistical feature-based methods attempt to use texture complexity [152], [155], motion complexity details [156], [157], combined with statistics from previously encoded blocks [158] to generate probabilistic models (e.g., Naive-Bayes) to early determine the best coding structure/parameters for a given image segment.…”

“…On the other hand, learning-based approaches use data sets composed of previously encoded image segments and associated encoding parameters to train machine learning models which can then be used to predict the coding structures/parameters for a given arbitrary content. Supervised learning algorithms such as Support Vector Machines (SVM) have been used predominantly in recent literature due to their less complexity and ability to handle binary classification effectively [154], [159]. In addition, techniques such as random forests [160], decision trees for data mining [161], and various deep learning-based methods [162], [163] have been attempted to predict coding parameters at various stages in the encoding…”

“…encoder optimization algorithms tend to provide engineering design parameters to trade-off the coding efficiency to the encoding complexity depending on the end-user's QoE requirements, network conditions, and computational resource constraints of the encoding servers [154], [156].…”

State-Of-The-Art In Video Processing: Compression, Optimization And Retrieval

“…Frame rate up conversion [11] enhances the original videos' temporal resolutions thus converting the frame rate between different systems by interpolation of frames between consecutive ones. Encoding process of HEVC inter prediction [14,20] computes quadtree partitioning of CTU and evaluates its hierarchy using rate distortion optimization.…”

A Motion Estimation based Algorithm for Encoding Time Reduction in HEVC

“…However, machine learning approaches are less suitable for real-time applications due to their high computational complexity. Erabadda 15 , et al use the SVMs to classify the CU. The SVM is trained by using the texture and context features of the coding unit.…”