Fully Connected Network-Based Intra Prediction for Image Coding

Li, Jiahao; Li, Bin; Xu, Jizheng; Xiong, Ruiqin; Gao, Wen

doi:10.1109/tip.2018.2817044

Cited by 167 publications

(115 citation statements)

References 31 publications

Supporting

Mentioning

110

Contrasting

Order By: Relevance

“…Unlike standard intra prediction in which the encoder chooses the best mode in a rate-distortion sense among several pre-defined modes, only one neural network among a set of neural networks does the prediction here. Unlike [14], our set contains both fully-connected and convolutional neural networks. This section first presents our set of neural networks.…”

Section: Predictionmentioning

confidence: 99%

“…Therefore, unlike the H.265 intra prediction modes, the convolutional neural networks of PNNS can model a large set of complex textures found in large image blocks. [14], called IPFCN-S, in terms of prediction PSNR. The 4 fully-connected neural networks in [14] predict image blocks of sizes 4 × 4, 8 × 8, 16 × 16, and 32 × 32 pixels respectively.…”

Section: E Relevance Of Convolutional Neural Network For Predictingmentioning

confidence: 99%

See 1 more Smart Citation

Context-Adaptive Neural Network-Based Prediction for Image Compression

Dumas

Roumy

Guillemot

2020

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

This paper describes a set of neural network architectures, called Prediction Neural Networks Set (PNNS), based on both fully-connected and convolutional neural networks, for intra image prediction. The choice of neural network for predicting a given image block depends on the block size, hence does not need to be signalled to the decoder. It is shown that, while fully-connected neural networks give good performance for small block sizes, convolutional neural networks provide better predictions in large blocks with complex textures. Thanks to the use of masks of random sizes during training, the neural networks of PNNS well adapt to the available context that may vary, depending on the position of the image block to be predicted. When integrating PNNS into a H.265 codec, PSNRrate performance gains going from 1.46% to 5.20% are obtained. These gains are on average 0.99% larger than those of prior neural network based methods. Unlike the H.265 intra prediction modes, which are each specialized in predicting a specific texture, the proposed PNNS can model a large set of complex textures.

show abstract

Section: Predictionmentioning

confidence: 99%

Section: E Relevance Of Convolutional Neural Network For Predictingmentioning

confidence: 99%

Context-Adaptive Neural Network-Based Prediction for Image Compression

Dumas

Roumy

Guillemot

2020

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…The final version of record is available at http://dx.doi.org/10.1109/TCSVT.2019.2954474 AV1 [141] ---3.0 --Note: 1 LL/RL/LH/RH denote low-delay low complexity, random-access low complexity, low-delay high-efficiency, and random-access high-efficiency configurations in early HM test conditions. [190][191][192][193], block up-sampling for intra frame coding [194], intra mode decision [195][196][197][198][199], transform [200], rate control [201,202], in-loop filtering/post-processing [203][204][205], arithmetic coding [206], or decoder-end artifact-removal and quality enhancement [207,208].…”

Section: Finer Precision Motion Estimation and Compensationmentioning

confidence: 99%

Recent Advances on HEVC Inter-Frame Coding: From Optimization to Implementation and Beyond

Zhang

Fan

et al. 2020

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

High Efficiency Video Coding (HEVC) has doubled the video compression ratio with equivalent subjective quality as compared to its predecessor H.264/AVC. The significant coding efficiency improvement is attributed to many new techniques. Interframe coding is one of the most powerful yet complicated techniques therein and has posed high computational burden thus main obstacle in HEVC-based real-time applications. Recently, plenty of research has been done to optimize the inter-frame coding, either to reduce the complexity for real-time applications, or to further enhance the encoding efficiency. In this paper, we provide a comprehensive review of the state-of-the-art techniques for HEVC inter-frame coding from three aspects, namely fast inter coding solutions, implementation on different hardware platforms as well as advanced inter coding techniques. More specifically, different algorithms in each aspect are further subdivided into sub-categories and compared in terms of pros, cons, coding efficiency and coding complexity. To the best of our knowledge, this is the first such comprehensive review of the recent advances of the inter-frame coding for HEVC and hopefully it would help the improvement, implementation and applications of HEVC as well as the ongoing development of the next generation video coding standard.

show abstract

“…Besides significantly improving the performances in these tasks, these works bring in new insights and methodologies for pixel-level motion modeling, which provide new foundations for the successive works. Meanwhile, more and more works [12][13][14][15] We follow both trends, deep learning-based motion modeling and deep learning-based video coding optimization, and offer optimized video coding techniques to better model the pixel-wise inconsistent displacement. Specifically, we choose to use deep learning techniques to interpolate a pixel-wise closer frame (PC-frame) from existing reconstructed frames.…”

Section: Introductionmentioning

confidence: 99%

Deep Reference Generation With Multi-Domain Hierarchical Constraints for Inter Prediction

Liu

Xia

Yang

2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Inter prediction is an important module in video coding for temporal redundancy removal, where similar reference blocks are searched from previously coded frames and employed to predict the block to be coded. Although traditional video codecs can estimate and compensate for block-level motions, their inter prediction performance is still heavily affected by the remaining inconsistent pixel-wise displacement caused by irregular rotation and deformation. In this paper, we address the problem by proposing a deep frame interpolation network to generate additional reference frames in coding scenarios. First, we summarize the previous adaptive convolutions used for frame interpolation and propose a factorized kernel convolutional network to improve the modeling capacity and simultaneously keep its compact form. Second, to better train this network, multi-domain hierarchical constraints are introduced to regularize the training of our factorized kernel convolutional network. For spatial domain, we use a gradually down-sampled and upsampled auto-encoder to generate the factorized kernels for frame interpolation at different scales. For quality domain, considering the inconsistent quality of the input frames, the factorized kernel convolution is modulated with quality-related features to learn to exploit more information from high quality frames. For frequency domain, a sum of absolute transformed difference loss that performs frequency transformation is utilized to facilitate network optimization from the view of coding performance. With the well-designed frame interpolation network regularized by multi-domain hierarchical constraints, our method surpasses HEVC on average 6.1% BD-rate saving and up to 11.0% BD-rate saving for the luma component under the random access configuration.

show abstract

Fully Connected Network-Based Intra Prediction for Image Coding

Cited by 167 publications

References 31 publications

Context-Adaptive Neural Network-Based Prediction for Image Compression

Context-Adaptive Neural Network-Based Prediction for Image Compression

Recent Advances on HEVC Inter-Frame Coding: From Optimization to Implementation and Beyond

Deep Reference Generation With Multi-Domain Hierarchical Constraints for Inter Prediction

Contact Info

Product

Resources

About