CNN-LSTM Learning Approach-Based Complexity Reduction for High-Efficiency Video Coding Standard

Abstract

High-Efficiency Video Coding provides a better compression ratio compared to earlier standard, H.264/Advanced Video Coding. In fact, HEVC saves 50% bit rate compared to H.264/AVC for the same subjective quality. This improvement is notably obtained through the hierarchical quadtree structured Coding Unit. However, the computational complexity significantly increases due to the full search Rate-Distortion Optimization, which allows reaching the optimal Coding Tree Unit partition. Despite the many speedup algorithms developed in the literature, the HEVC encoding complexity still remains a crucial problem in video coding field. Towards this goal, we propose in this paper a deep learning model-based fast mode decision algorithm for HEVC intermode. Firstly, we provide a deep insight overview of the proposed CNN-LSTM, which plays a kernel and pivotal role in this contribution, thus predicting the CU splitting and reducing the HEVC encoding complexity. Secondly, a large training and inference dataset for HEVC intercoding was investigated to train and test the proposed deep framework. Based on this framework, the temporal correlation of the CU partition for each video frame is solved by the LSTM network. Numerical results prove that the proposed CNN-LSTM scheme reduces the encoding complexity by 58.60% with an increase in the BD rate of 1.78% and a decrease in the BD-PSNR of -0.053 dB. Compared to the related works, the proposed scheme has achieved a best compromise between RD performance and complexity reduction, as proven by experimental results.

Deep Learning Based Low-Complexity andHigh-Speed Coding Unit Partition Decisionfor High Efficiency Video Coding

“…Soulef Bouaafi and Randa Khemiri et al (21) proposed a fast coding unit partition algorithm for HEVC inter-mode based on a deep learning model. The proposed CNN-LSTM, which is the core of this work, predicts CU splitting and simplifies HEVC encoding.…”

“…Numerous models and techniques have been proposed in the literature to reduce the complexity of the coding units and the bit rate ( 10)- (21). However, majority of the works showing that improvement in one or more parameter by sacrificing other parameter(s).…”

“…LeNet-5 has a bit rate of 0.61%; this shows that the effectiveness of compression is unaffected by the complexity reduction achieved by this method. However, the improvements made with ( 15),( 16),( 17),( 18),( 19), (24), and ( 27) are at the expense of a decrease in compression efficiency that reached 1.50%, 1.70%, and 1.29%, 0.88%, 1.75%, 1.70 53.00 [16] 1.29 55.51 [17] 0.88 57.21 [18] 1.75 71.20 [19] 1.38 69.06 [24] 1.50 51.90 [27] 1.78 58.60…”

VVC In‐Loop Filtering Based on Deep Convolutional Neural Network

“…In recent years, artificial intelligence has seen tremendous progress in computer vision topics, in particular in image and video compression [13][14][15]. Deep learning networks have been applied to enhance coding tools for HEVC and VVC standards, including intra and interprediction, transformation, quantization, and loop filtering [16,17]. With regards to the HEVC, Bouaafia et al in [14] proposed a reduction of HEVC complexity based on machine learning in the process of interprediction, which saves a good performance in terms of RD cost and computational complexity.…”

Complexity Analysis of New Future Video Coding (FVC) Standard Technology

“…When comparing our work to the state-of-the-art approaches in [22,38,39], we can conclude that our proposed scheme performs better in terms of the encoding efficiency. [40,41]. In this context, all the modules will jointly learn through a single loss function, in which they will collaborate with each other by considering the trade-off between reducing the number of compression bits and improving the quality of the decoded video.…”